Opened 6 years ago

Closed 5 years ago

#6913 closed defect (fixed)

24h test pattern encoding to h264 nut results in corrupted decoding

Reported by: fiuqwb Owned by:
Priority: important Component: avformat
Version: git-master Keywords: nut
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug: I encode a 24h file with FFmpeg, using synthetic content (test pattern and generated audio). In the latter part of the file, content fails to correctly play back using FFplay and VLC.

How to reproduce:

ffmpeg -f lavfi -i "aevalsrc='0.1*sin(2*PI*((90+mod(floor(t), 5)*30)-60/2)*t) | 0.1*sin(2*PI*((90+mod(floor(t), 5)*30)+60/2)*t):s=48000'" -f lavfi -i "testsrc2=r=30:s=1920x1080" -acodec aac -ab 64k -t 86400 -vcodec libx264 -tune animation -y TestPattern.nut

ffmpeg version N-89467-g0e5260226a Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-amf --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      56.  5.100 / 56.  5.100
  libavcodec     58.  6.103 / 58.  6.103
  libavformat    58.  3.100 / 58.  3.100
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  7.100 /  7.  7.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, lavfi, from 'aevalsrc='0.1*sin(2*PI*((90+mod(floor(t), 5)*30)-60/2)*t) | 0.1*sin(2*PI*((90+mod(floor(t), 5)*30)+60/2)*t):s=48000'':
  Duration: N/A, start: 0.000000, bitrate: 6144 kb/s
    Stream #0:0: Audio: pcm_f64le, 48000 Hz, stereo, dbl, 6144 kb/s
Input #1, lavfi, from 'testsrc2=r=30:s=1920x1080':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 30 tbr, 30 tbn, 30 tbc
Stream mapping:
  Stream #1:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
  Stream #0:0 -> #0:1 (pcm_f64le (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 000001b715aff300] using SAR=1/1
[libx264 @ 000001b715aff300] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 000001b715aff300] profile High, level 5.0
[libx264 @ 000001b715aff300] 264 - core 152 r2851 ba24899 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=6 deblock=1:1:1 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=0.40:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=5 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:0.60
Output #0, nut, to 'TestPattern.nut':
  Metadata:
    encoder         : Lavf58.3.100
    Stream #0:0: Video: h264 (libx264) (H264 / 0x34363248), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 30 fps, 61440 tbn, 30 tbc
    Metadata:
      encoder         : Lavc58.6.103 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1: Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, fltp, 64 kb/s
    Metadata:
      encoder         : Lavc58.6.103 aac
frame=2592000 fps= 77 q=-1.0 Lsize=70971382kB time=24:00:00.00 bitrate=6729.1kbits/s speed=2.57x
video:70189533kB audio:687798kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.132696%
[libx264 @ 000001b715aff300] frame I:10370 Avg QP:14.03  size: 58810
[libx264 @ 000001b715aff300] frame P:1179164 Avg QP:25.70  size: 31304
[libx264 @ 000001b715aff300] frame B:1402466 Avg QP:30.21  size: 24494
[libx264 @ 000001b715aff300] consecutive B-frames: 13.4% 45.2% 11.5%  3.1% 23.8%  3.0%
[libx264 @ 000001b715aff300] mb I  I16..4: 80.9% 11.8%  7.3%
[libx264 @ 000001b715aff300] mb P  I16..4:  0.8%  2.9%  0.4%  P16..4:  4.3%  4.0%  2.3%  0.0%  0.0%    skip:85.3%
[libx264 @ 000001b715aff300] mb B  I16..4:  0.1%  0.2%  0.5%  B16..8:  7.0%  2.9%  1.0%  direct: 0.9%  skip:87.4%  L0:53.2% L1:34.9% BI:11.8%
[libx264 @ 000001b715aff300] 8x8 transform intra:54.8% inter:18.2%
[libx264 @ 000001b715aff300] coded y,uvDC,uvAC intra: 19.4% 24.8% 24.4% inter: 3.6% 6.3% 5.7%
[libx264 @ 000001b715aff300] i16 v,h,dc,p: 88% 10%  2%  0%
[libx264 @ 000001b715aff300] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  4% 13% 79%  1%  0%  0%  0%  1%  0%
[libx264 @ 000001b715aff300] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 15% 22% 31%  8%  4%  5%  5%  6%  4%
[libx264 @ 000001b715aff300] i8c dc,h,v,p: 83%  7%  9%  1%
[libx264 @ 000001b715aff300] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 000001b715aff300] ref P L0: 40.1%  7.8% 18.0% 12.5% 10.1%  9.2%  2.3%
[libx264 @ 000001b715aff300] ref B L0: 60.1% 17.9% 13.6%  7.0%  1.4%
[libx264 @ 000001b715aff300] ref B L1: 95.8%  4.2%
[libx264 @ 000001b715aff300] kb/s:6655.01
[aac @ 000001b715b02900] Qavg: 328.695

Playing back this using FFplay results in errors (both written to stderr and visual) all throughout the latter parts of the video.

ffplay TestPattern.nut
ffplay version N-89467-g0e5260226a Copyright (c) 2003-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-amf --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      56.  5.100 / 56.  5.100
  libavcodec     58.  6.103 / 58.  6.103
  libavformat    58.  3.100 / 58.  3.100
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  7.100 /  7.  7.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, nut, from 'TestPattern.nut': vq=    0KB sq=    0B f=0/0
  Metadata:
    encoder         : Lavf58.3.100
  Duration: 24:00:00.05, start: 0.045333, bitrate: 6729 kb/s
    Stream #0:0: Video: h264 (High) (H264 / 0x34363248), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 61440 tbn, 60 tbc
    Metadata:
      encoder         : Lavc58.6.103 libx264
    Stream #0:1: Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, fltp
    Metadata:
      encoder         : Lavc58.6.103 aac
Seek to 41% ( 9:51:45) of total duration (24:00:00)       B f=0/0
Seek to 49% (11:48:45) of total duration (24:00:00)       0B f=3/3
Seek to 52% (12:23:15) of total duration (24:00:00)       0B f=9/9
Seek to 63% (15:03:45) of total duration (24:00:00)       0B f=23/23
Seek to 78% (18:45:00) of total duration (24:00:00)       0B f=41/41
Seek to 81% (19:32:15) of total duration (24:00:00)       0B f=45/44
[nut @ 000001874288b680] frame size > 2max_distance and no checksum
[nut @ 000001874288b680] frame size > 2max_distance and no checksum
    Last message repeated 1 times
[h264 @ 0000018742928680] reference picture missing during reorder
    Last message repeated 1 times
[h264 @ 0000018742928680] Missing reference picture, default is 65546
    Last message repeated 1 times
[h264 @ 000001874ad745a0] Missing reference picture, default is 65558
[h264 @ 000001874ad74e40] Missing reference picture, default is 65562
[nut @ 000001874288b680] frame size > 2max_distance and no checksum
[h264 @ 000001874ad752a0] Missing reference picture, default is 65562
[h264 @ 0000018742927de0] reference picture missing during reorder44
    Last message repeated 2 times
[h264 @ 0000018742927de0] Missing reference picture, default is 65562
    Last message repeated 2 times
    Last message repeated 2 times
[h264 @ 0000018742928240]     Last message repeated 2 timesB f=45/44
reference picture missing during reorder
[nut @ 000001874288b680] [h264 @ 0000018742928240] frame size > 2max_distance and no checksum
reference picture missing during reorder
[h264 @ 0000018742928240] Missing reference picture, default is 65570
    Last message repeated 1 times
    Last message repeated 1 times  17KB vq=  807KB sq=    0B f=45/44
[h264 @ 0000018742928680] reference picture missing during reorder
[h264 @ 0000018742928680] Missing reference picture, default is 65572
[h264 @ 000001874ad745a0] reference picture missing during reorder44
[h264 @ 000001874ad745a0] Missing reference picture, default is 65574
[nut @ 000001874288b680] frame size > 2max_distance and no checksum4
[h264 @ 000001874ad74e40] mmco: unref short failuresq=    0B f=45/44
[nut @ 000001874288b680] frame size > 2max_distance and no checksum4
[nut @ 000001874288b680] frame size > 2max_distance and no checksum4
[h264 @ 000001874ad69020] reference picture missing during reorder44
    Last message repeated 1 times
[h264 @ 000001874ad69020] Missing reference picture, default is 65608
    Last message repeated 1 times
[nut @ 000001874288b680] frame size > 2max_distance and no checksum4
[h264 @ 000001874ad74e40] Missing reference picture, default is 65620
[h264 @ 0000018742927de0] Missing reference picture, default is 65624
[h264 @ 0000018742928240] Missing reference picture, default is 65624
[h264 @ 0000018742928680] Missing reference picture, default is 65624
    Last message repeated 1 times
[h264 @ 000001874ad58040] mmco: unref short failuresq=    0B f=45/44
[h264 @ 000001874ad69020] Missing reference picture, default is 65634
[h264 @ 000001874ad745a0] Missing reference picture, default is 65634
    Last message repeated 1 times
[h264 @ 000001874ad74a00] Missing reference picture, default is 65634
[h264 @ 000001874ad74e40] Missing reference picture, default is 65634
[h264 @ 000001874ad752a0] Missing reference picture, default is 65644
    Last message repeated 1 times
[h264 @ 0000018742927de0] Missing reference picture, default is 65644
    Last message repeated 1 times
[h264 @ 0000018742928240] Missing reference picture, default is 65644

OS: Windows 10 v1709

Playback glitches also reproduce in VLC.

Change History (13)

comment:1 by Carl Eugen Hoyos, 6 years ago

Is the issue also reproducible without using an external library (libx264)?
Is the issue also reproducible with a smaller resolution?
Is the issue also reproducible without an audio stream?

comment:2 by fiuqwb, 6 years ago

I will experiment and provide answers. Can you recommend a suitable alternative for libx264? I am not entirely aware of what codecs are external/internal.

comment:3 by Carl Eugen Hoyos, 6 years ago

-vcodec mpeg4 -qscale 2 is the most common alternative.

comment:4 by fiuqwb, 6 years ago

With no audio, with 256x144 resolution and lixb264: issue remains, though slightly different in appearance. Now there are pauses in playback (picture freezes for a second, FFplay console output keeps counting timestamp but no other numbers change) instead of visual garbage. No errors are printed to FFplay stderr.

Hypothesis: some threshold value is no longer exceeded due to smaller data size from smaller resolution, so the visible behavior changes but the internal issue creating the bad data remains.

With no audio, with 256x144 resolution and huffyuv: issue does not reproduce.

With no audio, with 256x144 resolution and -vcodec mpeg4 -qscale 2: issue does not reproduce.

I will also try larger resolution with mpeg4, to verify that a smaller data size does not simply hide the problem, though it will take a while to do the encode due to the larger size - will report back tomorrow on results for that.

It seems likely that the issue is related specifically to libx264. If the mpeg4 test with larger resolution also does not reproduce the issue, what would be the appropriate next steps here?

comment:5 by fiuqwb, 6 years ago

I confirm no glitches with full resolution and mpeg4 codec. It seems to be specific to using the libx264 codec.

I will next attempt to reproduce with x264 separately, so see if this is an x264 issue or something related to its integration in FFmpeg.

comment:6 by fiuqwb, 6 years ago

With no audio, with 256x144 resolution and standalone x264: issue does NOT reproduce.

I used the same input (as rawvideo piped from FFmpeg).

ffmpeg -f lavfi -i "testsrc2=r=30:s=256x144" -t 86400 -f rawvideo -pixel_format yuv420p pipe:1   | x264 --output by_x264.mp4 --input-res 256x144 --crf 23 --tune animation --profile high --fps 30 -
ffmpeg version N-89467-g0e5260226a Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-amf --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      56.  5.100 / 56.  5.100
  libavcodec     58.  6.103 / 58.  6.103
  libavformat    58.  3.100 / 58.  3.100
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  7.100 /  7.  7.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, lavfi, from 'testsrc2=r=30:s=256x144':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x144 [SAR 1:1 DAR 16:9], 30 tbr, 30 tbn, 30 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, rawvideo, to 'pipe:1':
  Metadata:
    encoder         : Lavf58.3.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x144 [SAR 1:1 DAR 16:9], q=2-31, 13271 kb/s, 30 fps, 30 tbn, 30 tbc
    Metadata:
      encoder         : Lavc58.6.103 rawvideo
raw [info]: 256x144p 0:0 @ 30/1 fps (cfr)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x264 [info]: profile High, level 1.2
frame=2592000 fps=1280 q=-0.0 Lsize=139968000kB time=24:00:00.00 bitrate=13271.0kbits/s speed=42.7x
video:139968000kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
x264 [info]: frame I:10368 Avg QP:19.14  size:  5099
x264 [info]: frame P:667317 Avg QP:26.03  size:  1486
x264 [info]: frame B:1914315 Avg QP:31.24  size:   736
x264 [info]: consecutive B-frames:  1.4% 13.4%  7.7% 24.3% 15.8% 37.3%
x264 [info]: mb I  I16..4: 36.5% 16.6% 46.9%
x264 [info]: mb P  I16..4:  2.6%  3.7%  1.3%  P16..4: 18.8% 13.6% 10.3%  0.0%  0.0%    skip:49.7%
x264 [info]: mb B  I16..4:  0.2%  0.2%  0.1%  B16..8: 23.2%  9.6%  2.9%  direct: 2.8%  skip:61.1%  L0:47.9% L1:44.5% BI: 7.6%
x264 [info]: 8x8 transform intra:43.3% inter:28.4%
x264 [info]: coded y,uvDC,uvAC intra: 16.1% 34.7% 30.0% inter: 7.4% 18.9% 15.3%
x264 [info]: i16 v,h,dc,p: 72% 22%  6%  0%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 11% 75%  1%  0%  0%  0%  0%  0%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 42% 25% 24%  2%  1%  1%  2%  2%  2%
x264 [info]: i8c dc,h,v,p: 42% 19% 38%  1%
x264 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x264 [info]: ref P L0: 53.1%  7.7% 16.2%  8.1%  6.8%  7.9%  0.3%
x264 [info]: ref B L0: 74.3% 13.5%  7.8%  3.2%  1.2%
x264 [info]: ref B L1: 94.7%  5.3%
x264 [info]: kb/s:227.27

encoded 2592000 frames, 1280.30 fps, 227.27 kb/s


x264 --version
x264 0.152.2851 ba24899
(libswscale 4.7.101)
(libavformat 57.75.100)
(ffmpegsource 2.23.0.0)
built by Komisar on Jul  2 2017, gcc: 4.9.2 (multilib.generic.Komisar)
x264 configuration: --bit-depth=8 --chroma-format=all
libx264 configuration: --bit-depth=8 --chroma-format=all
x264 license: GPL version 2 or later
libswscale/libavformat/ffmpegsource license: GPL version 2 or later

comment:7 by Carl Eugen Hoyos, 6 years ago

Component: undeterminedavformat
Keywords: nut added
Priority: normalimportant
Reproduced by developer: set
Summary: 24h test pattern encode results in corrupted output and fails to play correctly in ffplay24h test pattern encoding to h264 nut results in corrupted decoding
Version: unspecifiedgit-master

I can confirm that there is an issue, it is also reproducible when decoding the output file with v0.5. The issue is not seeking-related.
The issue is apparently nut-related, I don't know if it is an issue with the specification, our muxer or our demuxer. In any case, not FFplay-related.
I cannot test the demuxing issue with libnut (Bad NUT Version), nutindex happily writes an output file with identical (or at least very similar) issues.

Last edited 6 years ago by Carl Eugen Hoyos (previous) (diff)

comment:8 by Elon Musk, 5 years ago

What commands you used to reproduce this?[]

comment:9 by Elon Musk, 5 years ago

I think this one should be fixed. Try to reproduce bug with latest git master.

comment:10 by Carl Eugen Hoyos, 5 years ago

I was hoping you wouldn't ask me;-)

comment:11 by Carl Eugen Hoyos, 5 years ago

Will of course test.

comment:12 by Elon Musk, 5 years ago

I never asked specifically you... I doubt you can encode such large file in reasonable time.

comment:13 by Carl Eugen Hoyos, 5 years ago

Resolution: fixed
Status: newclosed

I can confirm that the issue was fixed by Paul in 6bfc935232fbdb584abef0ec4560be00c5797f46

Note: See TracTickets for help on using tickets.