Opened 7 years ago

Last modified 7 years ago

#5935 new defect

Duration glitch and seek offset problems with transcodes to h264/aac/mp4

Reported by: Neil Slater Owned by:
Priority: normal Component: undetermined
Version: git-master Keywords: duration h264 mov
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug: When transcoding content to h264/aac/mp4, ffmpeg writes timing metadata such that duration is longer than expected, and resulting videos do not seek accurately in HTML5 video player (Chrome and Firefox exhibit same behaviour). I have confirmed with alternative content that the HTML5 player does not always do this - it is specifically content transcoded using ffmpeg, and appears on *all* content transcoded to h264/aac/mp4. It does not appear on content without audio.

How to reproduce: As the fault is quite subtle in nature, I have put together a MCVE with a few variations at https://github.com/Neil-Aframe/ffmpeg_timing_examples - this includes sample output from ffmbc to same target format which does not exhibit the fault.

Below is typical command which results in fault:

%ffprobe outputs/ffmbc_h264_aac.mp4 2>&1 | grep "Duration"
  Duration: 00:00:02.40, start: 0.000000, bitrate: 204 kb/s

% ffmpeg -y -i outputs/ffmbc_h264_aac.mp4 -f mp4 -c:v h264 -c:a aac -vb 1M test.mp4
ffmpeg version git-2016-11-08-1bbb18f Copyright (c) 2000-2016 the FFmpeg developers
  built with Apple LLVM version 7.0.2 (clang-700.1.81)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD-1bbb18f --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-opencl --disable-lzma --enable-nonfree --enable-vda
  libavutil      55. 35.100 / 55. 35.100
  libavcodec     57. 66.101 / 57. 66.101
  libavformat    57. 57.100 / 57. 57.100
  libavdevice    57.  2.100 / 57.  2.100
  libavfilter     6. 66.100 /  6. 66.100
  libavresample   3.  2.  0 /  3.  2.  0
  libswscale      4.  3.100 /  4.  3.100
  libswresample   2.  4.100 /  2.  4.100
  libpostproc    54.  2.100 / 54.  2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'outputs/ffmbc_h264_aac.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isomiso2avc1mp41
    creation_time   : 2016-11-09T14:35:58.000000Z
    encoder         : FFmbc 0.7
  Duration: 00:00:02.40, start: 0.000000, bitrate: 204 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 690x360 [SAR 1:1 DAR 23:12], 129 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc (default)
    Metadata:
      creation_time   : 2016-11-09T14:35:58.000000Z
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 62 kb/s (default)
    Metadata:
      creation_time   : 2016-11-09T14:35:58.000000Z
      handler_name    : SoundHandler
[libx264 @ 0x7fd035800c00] using SAR=1/1
[libx264 @ 0x7fd035800c00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x7fd035800c00] profile High, level 3.0
[libx264 @ 0x7fd035800c00] 264 - core 148 r2699 a5e06b9 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=1000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'test.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.57.100
    Stream #0:0(und): Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 690x360 [SAR 1:1 DAR 23:12], q=-1--1, 1000 kb/s, 25 fps, 12800 tbn, 25 tbc (default)
    Metadata:
      creation_time   : 2016-11-09T14:35:58.000000Z
      handler_name    : VideoHandler
      encoder         : Lavc57.66.101 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/1000000 buffer size: 0 vbv_delay: -1
    Stream #0:1(und): Audio: aac (LC) ([64][0][0][0] / 0x0040), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2016-11-09T14:35:58.000000Z
      handler_name    : SoundHandler
      encoder         : Lavc57.66.101 aac
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
frame=   60 fps=0.0 q=-1.0 Lsize=     143kB time=00:00:02.43 bitrate= 480.3kbits/s speed=5.24x
video:102kB audio:38kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.314983%
[libx264 @ 0x7fd035800c00] frame I:1     Avg QP:15.49  size: 20251
[libx264 @ 0x7fd035800c00] frame P:16    Avg QP: 6.87  size:  4051
[libx264 @ 0x7fd035800c00] frame B:43    Avg QP:16.15  size:   438
[libx264 @ 0x7fd035800c00] consecutive B-frames:  1.7%  6.7%  5.0% 86.7%
[libx264 @ 0x7fd035800c00] mb I  I16..4:  5.1% 77.3% 17.6%
[libx264 @ 0x7fd035800c00] mb P  I16..4:  0.1%  0.1%  0.3%  P16..4: 24.5%  1.2%  2.9%  0.0%  0.0%    skip:70.8%
[libx264 @ 0x7fd035800c00] mb B  I16..4:  0.1%  0.0%  0.1%  B16..8:  9.0%  0.4%  0.4%  direct: 0.3%  skip:89.8%  L0:17.7% L1:81.6% BI: 0.7%
[libx264 @ 0x7fd035800c00] final ratefactor: 7.05
[libx264 @ 0x7fd035800c00] 8x8 transform intra:69.4% inter:39.7%
[libx264 @ 0x7fd035800c00] coded y,uvDC,uvAC intra: 82.6% 82.8% 79.5% inter: 3.8% 4.1% 3.4%
[libx264 @ 0x7fd035800c00] i16 v,h,dc,p: 34% 56%  7%  3%
[libx264 @ 0x7fd035800c00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 32% 13% 15%  5%  7%  9%  5%  9%  6%
[libx264 @ 0x7fd035800c00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 37% 19% 11%  6%  7%  7%  5%  5%  3%
[libx264 @ 0x7fd035800c00] i8c dc,h,v,p: 43% 20% 29%  9%
[libx264 @ 0x7fd035800c00] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x7fd035800c00] ref P L0: 93.4%  1.0%  4.0%  1.6%
[libx264 @ 0x7fd035800c00] ref B L0: 73.2% 22.6%  4.2%
[libx264 @ 0x7fd035800c00] ref B L1: 89.1% 10.9%
[libx264 @ 0x7fd035800c00] kb/s:346.36

% ffprobe test.mp4 2>&1 | grep "Duration"
  Duration: 00:00:02.44, start: 0.000000, bitrate: 480 kb/s

In the above case, the output file reports incorrect duration, but otherwise plays and seeks to correct frame OK. This is an exception to what I usually get with production data though, which has same duration glitch but also lags a frame, sometimes 2 frames, when HTML5 video player seeks to a specific time.

In production, the fault is complicated with other issues (to do with H264 GOP settings). However, the following is common to almost all production inputs I have tried:

  • ffmbc output does not exhibit the fault. Duration and player seek are accurate.


  • Latest ffmpeg version transcoding to vp8/vorbis/webm usually has desired correct behaviour, accurate duration and no seek problems. It doesn't for unknown reason in my test example.

It is the success of the alternatives (along with the odd duration reported) which makes me think this is a fault in ffmpeg and not in HTML5 player.

In addition, I have tried (but not logged in detail) all available h264 encoder libraries, quite a few variations of h264 flags, and also confirmed the fault (or maybe a similar one) appears in versions 2.7.2 and 2.6.1

Attachments (1)

ffmpeg-20161109-161805.log (162.2 KB ) - added by Neil Slater 7 years ago.
Log of transcode with both duration and frame offset problem

Download all attachments as: .zip

Change History (5)

by Neil Slater, 7 years ago

Attachment: ffmpeg-20161109-161805.log added

Log of transcode with both duration and frame offset problem

comment:1 by Neil Slater, 7 years ago

I will try to identify a source video (as opposed to separate image frames and audio) with timecode burn-in that I can share as well to better demonstrate the issue as I am facing it in production.

comment:2 by Neil Slater, 7 years ago

Update: I added a few more transcodes to the demo project. In addition I noted that when transcoding from "broken" ffmpeg version (1 frame out and wrong duration) using ffmbc to re-transcode, then the resulting file has correct duration and seeks perfectly in the HTML5 player.

If I try to use the "copy" codecs, as following to investigate whether this is logic mainly in the mp4 container or the stream transcoding:

ffmbc -y -i encoded_by_ffmpeg.mp4 -f mp4 -vcodec copy -acodec copy test.mp4

Then ffmbc returns an error:

[mp4 @ 0x7ffbc300a200] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 25600 >= 25600
av_interleaved_write_frame(): Invalid argument

I don't know how to interpret the error message, nor whether this is a limitation of ffmbc or a pointer to something relevant to the problem.

Last edited 7 years ago by Neil Slater (previous) (diff)

comment:3 by Carl Eugen Hoyos, 7 years ago

Keywords: h264 mov added; timing removed

Is the issue also reproducible if you use another audio codec?

comment:4 by Neil Slater, 7 years ago

@cehoyos: Yes, the problem also occurs using mp3 audio codec, almost identical to aac (duration error is slightly different, seek offset error is same). I have added that example to the demonstration repo: https://github.com/Neil-Aframe/ffmpeg_timing_examples/blob/master/test_ffmpeg_h264_mp3_mp4.html

Note: See TracTickets for help on using tickets.