Opened 3 years ago

Last modified 3 years ago

#4260 new defect

hqdn3d usage in segment input creation creates invalid last segment

Reported by: blacktrash Owned by:
Priority: normal Component: avfilter
Version: git-master Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:
With some NTSC frame rate input conversion using the hqdn3d filter and then hls segmenting the result an invalid last segment without video is created.

How to reproduce:
The following boiled down command creates 5 valid segments if the hqdn3d filter is ommitted, and 6 with hqdn3d (the last one invalid):

$ ffmpeg -y -i hqdn3d-test.mov -filter:v hqdn3d,scale=480:270,fps=30000/1001 -filter:a aresample=osr=44100 -f mpegts -c:v libx264 -profile:v baseline -x264opts level=3.1:ref=1 -force_key_frames 'expr:gte(t,n_forced*5)' -c:a libfdk_aac -b:a 64k - | ffmpeg -i - -c copy -map 0 -f segment -segment_list_type hls -segment_time 5 -segment_time_delta `awk 'BEGIN { print 1 / (2 * 30000/1001) }'` -segment_list test.m3u8 %d.ts
ffmpeg version N-68781-ge405a8a Copyright (c) 2000-2014 the FFmpeg developers
  built on Dec 30 2014 11:15:15 with Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)
  configuration: --enable-gpl --enable-version3 --enable-nonfree --disable-ffserver --enable-shared --enable-libfdk-aac --enable-libx265 --enable-libx264 --enable-libxvid --enable-libfaac --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libvidstab --extra-cflags=-I/sw/include --extra-libs=-L/sw/lib --enable-openssl
  libavutil      54. 15.100 / 54. 15.100
  libavcodec     56. 19.100 / 56. 19.100
  libavformat    56. 16.102 / 56. 16.102
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  6.100 /  5.  6.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
ffmpeg version N-68781-ge405a8a Copyright (c) 2000-2014 the FFmpeg developers
  built on Dec 30 2014 11:15:15 with Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)
  configuration: --enable-gpl --enable-version3 --enable-nonfree --disable-ffserver --enable-shared --enable-libfdk-aac --enable-libx265 --enable-libx264 --enable-libxvid --enable-libfaac --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libvidstab --extra-cflags=-I/sw/include --extra-libs=-L/sw/lib --enable-openssl
  libavutil      54. 15.100 / 54. 15.100
  libavcodec     56. 19.100 / 56. 19.100
  libavformat    56. 16.102 / 56. 16.102
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  6.100 /  5.  6.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'hqdn3d-test.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 537199360
    compatible_brands: qt
    creation_time   : 2014-04-15 04:54:41
  Duration: 00:00:30.03, start: 0.000000, bitrate: 165031 kb/s
    Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080, 163491 kb/s, SAR 1920:1920 DAR 16:9, 29.97 fps, 29.97 tbr, 30k tbn, 60k tbc (default)
    Metadata:
      creation_time   : 2014-04-15 04:54:41
      handler_name    : Apple Alias Data Handler
      encoder         : H.264
      timecode        : 00:00:00;00
    Stream #0:1(eng): Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, stereo, s16, 1536 kb/s (default)
    Metadata:
      creation_time   : 2014-04-15 04:54:41
      handler_name    : Apple Alias Data Handler
      timecode        : 00:00:00;00
    Stream #0:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
    Metadata:
      creation_time   : 2014-04-15 05:05:56
      handler_name    : Apple Alias Data Handler
      timecode        : 00:00:00;00
[libx264 @ 0x7fb2f4829e00] using SAR=1/1
[libx264 @ 0x7fb2f4829e00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x7fb2f4829e00] profile Constrained Baseline, level 3.1
Output #0, mpegts, to 'pipe:':
  Metadata:
    major_brand     : qt
    minor_version   : 537199360
    compatible_brands: qt
    encoder         : Lavf56.16.102
    Stream #0:0(eng): Video: h264 (libx264), yuv420p, 480x270 [SAR 1:1 DAR 16:9], q=-1--1, 29.97 fps, 90k tbn, 29.97 tbc (default)
    Metadata:
      creation_time   : 2014-04-15 04:54:41
      handler_name    : Apple Alias Data Handler
      timecode        : 00:00:00;00
      encoder         : Lavc56.19.100 libx264
    Stream #0:1(eng): Audio: aac (libfdk_aac), 44100 Hz, stereo, s16, 64 kb/s (default)
    Metadata:
      creation_time   : 2014-04-15 04:54:41
      handler_name    : Apple Alias Data Handler
      timecode        : 00:00:00;00
      encoder         : Lavc56.19.100 libfdk_aac
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (pcm_s16le (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
Input #0, mpegts, from 'pipe:':=     425kB time=00:00:05.94 bitrate= 586.2kbits/s
  Duration: N/A, start: 1.400000, bitrate: N/A
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 480x270 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Stream #0:1[0x101](eng): Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 67 kb/s
Output #0, segment, to '%d.ts':
  Metadata:
    encoder         : Lavf56.16.102
    Stream #0:0: Video: h264 ([27][0][0][0] / 0x001B), yuv420p, 480x270 [SAR 1:1 DAR 16:9], q=2-31, 29.97 fps, 29.97 tbr, 90k tbn, 29.97 tbc
    Stream #0:1(eng): Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, 67 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
frame=  900 fps= 72 q=-1.0 Lsize=    3419kB time=00:00:30.04 bitrate= 932.2kbits/s
video:2820kB audio:236kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 11.871079%
[libx264 @ 0x7fb2f4829e00] frame I:21    Avg QP:22.38  size: 17633
[libx264 @ 0x7fb2f4829e00] frame P:879   Avg QP:25.79  size:  2864
[libx264 @ 0x7fb2f4829e00] mb I  I16..4: 23.6%  0.0% 76.4%
[libx264 @ 0x7fb2f4829e00] mb P  I16..4:  3.4%  0.0%  3.0%  P16..4: 35.7% 15.1%  5.2%  0.0%  0.0%    skip:37.5%
[libx264 @ 0x7fb2f4829e00] coded y,uvDC,uvAC intra: 49.6% 68.7% 34.7% inter: 22.6% 23.1% 4.2%
[libx264 @ 0x7fb2f4829e00] i16 v,h,dc,p: 19% 38%  9% 33%
[libx264 @ 0x7fb2f4829e00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 31% 14%  6%  6%  6%  7%  5%  6%
[libx264 @ 0x7fb2f4829e00] i8c dc,h,v,p: 49% 30% 12%  9%
[libx264 @ 0x7fb2f4829e00] kb/s:769.35
frame=  900 fps= 93 q=-1.0 Lsize=N/A time=00:00:30.09 bitrate=N/A
video:2826kB audio:236kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
$ ffprobe 6.ts
ffprobe version N-68781-ge405a8a Copyright (c) 2007-2014 the FFmpeg developers
  built on Dec 30 2014 11:15:15 with Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)
  configuration: --enable-gpl --enable-version3 --enable-nonfree --disable-ffserver --enable-shared --enable-libfdk-aac --enable-libx265 --enable-libx264 --enable-libxvid --enable-libfaac --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libvidstab --extra-cflags=-I/sw/include --extra-libs=-L/sw/lib --enable-openssl
  libavutil      54. 15.100 / 54. 15.100
  libavcodec     56. 19.100 / 56. 19.100
  libavformat    56. 16.102 / 56. 16.102
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  6.100 /  5.  6.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
[mpeg @ 0x7fa46880a000] Format mpeg detected only with low score of 25, misdetection possible!
[mp2 @ 0x7fa468826200] Header missing
[mpeg @ 0x7fa46880a000] decoding for stream 0 failed
[mpeg @ 0x7fa46880a000] Could not find codec parameters for stream 0 (Audio: mp2, 0 channels, s16p): unspecified frame size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
6.ts: End of file

Unfortunately - for reproduction - this happens rather rarely. The smallest input sample I came across so far is 591MB. It was used for the above. I can provide it for download if it is too large to attach here. It can't be shortened because the issue depends on exact duration as well.
The issue is reliably replicable also with intermediate mp4 (and h264_mp4toannexb filter for segmenting) or mpegts intermediate files.
In practice this can be worked around by just omitting manually editing the m3u8 file to omit the last segment. The stream then works ok. But this is obviously less than optimal.

Attachments (2)

hqdn3d-test.mov (2.0 MB) - added by blacktrash 3 years ago.
hqdn3d-test-a.mov (2.3 MB) - added by blacktrash 3 years ago.
input sample with aac (native ffmpeg encoder) audio track

Change History (5)

comment:1 Changed 3 years ago by blacktrash

591MB is too large to attach, I will make the sample available for download later.

comment:2 Changed 3 years ago by blacktrash

I was able to create a small input sample (exact same duration) with which the issue can be reproduced (no audio).

I've compacted the triggering command line to (no audio needed if -segment_time_delta is used, see below):

$ ffmpeg -y -i hqdn3d-test.mov -an -filter:v hqdn3d,scale=480:270 \
-f mpegts -c:v libx264 \
-profile:v baseline -x264opts level=3.1:ref=1 \
-force_key_frames 'expr:gte(t,n_forced*5)' - \
| ffmpeg -i - -c copy -map 0 -f segment -segment_time 5 \
-segment_time_delta `awk 'BEGIN { print 1 / (2 * 30000/1001) }'` \
-segment_list test.m3u8 %d.ts

Or, with hardcoded awk calculation result:

$ ffmpeg -y -i hqdn3d-test.mov -an -filter:v hqdn3d,scale=480:270 \
-f mpegts -c:v libx264 \
-profile:v baseline -x264opts level=3.1:ref=1 \
-force_key_frames 'expr:gte(t,n_forced*5)' - \
| ffmpeg -i - -c copy -map 0 -f segment -segment_time 5 \
-segment_time_delta 0.0166833 \
-segment_list test.m3u8 %d.ts

Recap of triggering conditions:

  • hqdn3d filter in mpegts creation
  • constrained baseline profile in mpegts creation
  • precise recommended segment_time_delta (half framerate) or an aac audio track (for the latter, see https://trac.ffmpeg.org/ticket/3859 probably)
Last edited 3 years ago by blacktrash (previous) (diff)

Changed 3 years ago by blacktrash

Changed 3 years ago by blacktrash

input sample with aac (native ffmpeg encoder) audio track

comment:3 Changed 3 years ago by blacktrash

I've attached a sample with aac audio, created with

$ ffmpeg -i real-world-sample.mov -filter:v scale=480:270 \
-strict -2 -c:a aac -b:a 64k hqdn3d-test-a.mov

Using that as input the issue can be reproduced with:

$ ffmpeg -i hqdn3d-test-a.mov -filter:v hqdn3d -f mpegts \
-c:v libx264 -profile:v baseline -x264opts level=3.1:ref=1 \
-force_key_frames 'expr:gte(t,n_forced*5)' -c:a copy - \
| ffmpeg -i - -c copy -map 0 -f segment -segment_time 5 \
-segment_list test.m3u8 %d.ts

i.e., native ffmpeg aac, and in this particular case even without -segment_time_delta.

Note: See TracTickets for help on using tickets.