Opened 7 years ago
Last modified 6 years ago
#5935 new defect
Duration glitch and seek offset problems with transcodes to h264/aac/mp4
|Reported by:||Neil Slater||Owned by:|
|Version:||git-master||Keywords:||duration h264 mov|
|Blocking:||Reproduced by developer:||no|
|Analyzed by developer:||no|
Summary of the bug: When transcoding content to h264/aac/mp4, ffmpeg writes timing metadata such that duration is longer than expected, and resulting videos do not seek accurately in HTML5 video player (Chrome and Firefox exhibit same behaviour). I have confirmed with alternative content that the HTML5 player does not always do this - it is specifically content transcoded using ffmpeg, and appears on *all* content transcoded to h264/aac/mp4. It does not appear on content without audio.
How to reproduce: As the fault is quite subtle in nature, I have put together a MCVE with a few variations at https://github.com/Neil-Aframe/ffmpeg_timing_examples - this includes sample output from ffmbc to same target format which does not exhibit the fault.
Below is typical command which results in fault:
%ffprobe outputs/ffmbc_h264_aac.mp4 2>&1 | grep "Duration" Duration: 00:00:02.40, start: 0.000000, bitrate: 204 kb/s % ffmpeg -y -i outputs/ffmbc_h264_aac.mp4 -f mp4 -c:v h264 -c:a aac -vb 1M test.mp4 ffmpeg version git-2016-11-08-1bbb18f Copyright (c) 2000-2016 the FFmpeg developers built with Apple LLVM version 7.0.2 (clang-700.1.81) configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD-1bbb18f --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-opencl --disable-lzma --enable-nonfree --enable-vda libavutil 55. 35.100 / 55. 35.100 libavcodec 57. 66.101 / 57. 66.101 libavformat 57. 57.100 / 57. 57.100 libavdevice 57. 2.100 / 57. 2.100 libavfilter 6. 66.100 / 6. 66.100 libavresample 3. 2. 0 / 3. 2. 0 libswscale 4. 3.100 / 4. 3.100 libswresample 2. 4.100 / 2. 4.100 libpostproc 54. 2.100 / 54. 2.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'outputs/ffmbc_h264_aac.mp4': Metadata: major_brand : mp42 minor_version : 0 compatible_brands: isomiso2avc1mp41 creation_time : 2016-11-09T14:35:58.000000Z encoder : FFmbc 0.7 Duration: 00:00:02.40, start: 0.000000, bitrate: 204 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 690x360 [SAR 1:1 DAR 23:12], 129 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc (default) Metadata: creation_time : 2016-11-09T14:35:58.000000Z handler_name : VideoHandler Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 62 kb/s (default) Metadata: creation_time : 2016-11-09T14:35:58.000000Z handler_name : SoundHandler [libx264 @ 0x7fd035800c00] using SAR=1/1 [libx264 @ 0x7fd035800c00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX [libx264 @ 0x7fd035800c00] profile High, level 3.0 [libx264 @ 0x7fd035800c00] 264 - core 148 r2699 a5e06b9 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=1000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, mp4, to 'test.mp4': Metadata: major_brand : mp42 minor_version : 0 compatible_brands: isomiso2avc1mp41 encoder : Lavf57.57.100 Stream #0:0(und): Video: h264 (libx264) ( / 0x0021), yuv420p, 690x360 [SAR 1:1 DAR 23:12], q=-1--1, 1000 kb/s, 25 fps, 12800 tbn, 25 tbc (default) Metadata: creation_time : 2016-11-09T14:35:58.000000Z handler_name : VideoHandler encoder : Lavc57.66.101 libx264 Side data: cpb: bitrate max/min/avg: 0/0/1000000 buffer size: 0 vbv_delay: -1 Stream #0:1(und): Audio: aac (LC) ( / 0x0040), 44100 Hz, stereo, fltp, 128 kb/s (default) Metadata: creation_time : 2016-11-09T14:35:58.000000Z handler_name : SoundHandler encoder : Lavc57.66.101 aac Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264)) Stream #0:1 -> #0:1 (aac (native) -> aac (native)) Press [q] to stop, [?] for help frame= 60 fps=0.0 q=-1.0 Lsize= 143kB time=00:00:02.43 bitrate= 480.3kbits/s speed=5.24x video:102kB audio:38kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.314983% [libx264 @ 0x7fd035800c00] frame I:1 Avg QP:15.49 size: 20251 [libx264 @ 0x7fd035800c00] frame P:16 Avg QP: 6.87 size: 4051 [libx264 @ 0x7fd035800c00] frame B:43 Avg QP:16.15 size: 438 [libx264 @ 0x7fd035800c00] consecutive B-frames: 1.7% 6.7% 5.0% 86.7% [libx264 @ 0x7fd035800c00] mb I I16..4: 5.1% 77.3% 17.6% [libx264 @ 0x7fd035800c00] mb P I16..4: 0.1% 0.1% 0.3% P16..4: 24.5% 1.2% 2.9% 0.0% 0.0% skip:70.8% [libx264 @ 0x7fd035800c00] mb B I16..4: 0.1% 0.0% 0.1% B16..8: 9.0% 0.4% 0.4% direct: 0.3% skip:89.8% L0:17.7% L1:81.6% BI: 0.7% [libx264 @ 0x7fd035800c00] final ratefactor: 7.05 [libx264 @ 0x7fd035800c00] 8x8 transform intra:69.4% inter:39.7% [libx264 @ 0x7fd035800c00] coded y,uvDC,uvAC intra: 82.6% 82.8% 79.5% inter: 3.8% 4.1% 3.4% [libx264 @ 0x7fd035800c00] i16 v,h,dc,p: 34% 56% 7% 3% [libx264 @ 0x7fd035800c00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 32% 13% 15% 5% 7% 9% 5% 9% 6% [libx264 @ 0x7fd035800c00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 37% 19% 11% 6% 7% 7% 5% 5% 3% [libx264 @ 0x7fd035800c00] i8c dc,h,v,p: 43% 20% 29% 9% [libx264 @ 0x7fd035800c00] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fd035800c00] ref P L0: 93.4% 1.0% 4.0% 1.6% [libx264 @ 0x7fd035800c00] ref B L0: 73.2% 22.6% 4.2% [libx264 @ 0x7fd035800c00] ref B L1: 89.1% 10.9% [libx264 @ 0x7fd035800c00] kb/s:346.36 % ffprobe test.mp4 2>&1 | grep "Duration" Duration: 00:00:02.44, start: 0.000000, bitrate: 480 kb/s
In the above case, the output file reports incorrect duration, but otherwise plays and seeks to correct frame OK. This is an exception to what I usually get with production data though, which has same duration glitch but also lags a frame, sometimes 2 frames, when HTML5 video player seeks to a specific time.
In production, the fault is complicated with other issues (to do with H264 GOP settings). However, the following is common to almost all production inputs I have tried:
- ffmbc output does not exhibit the fault. Duration and player seek are accurate.
- Latest ffmpeg version transcoding to vp8/vorbis/webm usually has desired correct behaviour, accurate duration and no seek problems. It doesn't for unknown reason in my test example.
It is the success of the alternatives (along with the odd duration reported) which makes me think this is a fault in ffmpeg and not in HTML5 player.
In addition, I have tried (but not logged in detail) all available h264 encoder libraries, quite a few variations of h264 flags, and also confirmed the fault (or maybe a similar one) appears in versions 2.7.2 and 2.6.1
Change History (5)
by , 7 years ago
comment:1 by , 7 years ago
I will try to identify a source video (as opposed to separate image frames and audio) with timecode burn-in that I can share as well to better demonstrate the issue as I am facing it in production.
comment:2 by , 7 years ago
Update: I added a few more transcodes to the demo project. In addition I noted that when transcoding from "broken" ffmpeg version (1 frame out and wrong duration) using ffmbc to re-transcode, then the resulting file has correct duration and seeks perfectly in the HTML5 player.
If I try to use the "copy" codecs, as following to investigate whether this is logic mainly in the mp4 container or the stream transcoding:
ffmbc -y -i encoded_by_ffmpeg.mp4 -f mp4 -vcodec copy -acodec copy test.mp4
Then ffmbc returns an error:
[mp4 @ 0x7ffbc300a200] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 25600 >= 25600 av_interleaved_write_frame(): Invalid argument
I don't know how to interpret the error message, nor whether this is a limitation of ffmbc or a pointer to something relevant to the problem.
comment:3 by , 6 years ago
|Keywords:||h264 mov added; timing removed|
Is the issue also reproducible if you use another audio codec?
comment:4 by , 6 years ago
@cehoyos: Yes, the problem also occurs using mp3 audio codec, almost identical to aac (duration error is slightly different, seek offset error is same). I have added that example to the demonstration repo: https://github.com/Neil-Aframe/ffmpeg_timing_examples/blob/master/test_ffmpeg_h264_mp3_mp4.html
Log of transcode with both duration and frame offset problem