Opened 3 months ago

Last modified 3 months ago

#9484 new defect

HLS muxer: inital fmp4 segment has non-zero duration even if `hls_init_time` is set to zero

Reported by: kebl Owned by:
Priority: normal Component: undetermined
Version: unspecified Keywords: hls ffmpeg
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

The HLS muxer supports fragmented mp4, which by default produces an initial segment called init.mp4. With ffmpeg from the Arch repo (version n4.4) the command below produces an initial segment 6.08 seconds long, but with static builds of ffmpeg 4.4.1 and 4.2.2 the initial segment has no duration. hls_init_time is set to zero so the inital segment should have no duration.

The effect the non-zero duration has is concatenating segments doesn't work properly anymore. For example with a no-duration initial segment, $ cat init.mp4 out2.m4s out3.m4s | ffplay - will play the segments starting from out2.m4s (12 seconds in from the input video). But with a non-zero duration it'll play the first six seconds of the input video, freeze-frame for six seconds, then play out2.m4s. (It looks like the initial segment has the same video content as the 0th segment because $ cat init.mp4 | ffplay - behaves identically to $ cat init.mp4 out0.m4s | ffplay -.

% ffmpeg -i in.webm -g 24 -sc_threshold 0 -f hls -hls_init_time 0 -hls_time 6 -hls_segment_type fmp4 out.m3u8

in.webm is https://upload.wikimedia.org/wikipedia/commons/e/e7/Slow_motion_drop.webm.


Initial segment produced by Arch ffmpeg (Duration: 00:00:06.08):

% ffprobe init.mp4
ffprobe version n4.4 Copyright (c) 2007-2021 the FFmpeg developers
  built with gcc 11.1.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'init.mp4':
  Metadata:
    major_brand     : iso5
    minor_version   : 512
    compatible_brands: iso5iso6mp41
    encoder         : Lavf58.76.100
  Duration: 00:00:06.08, start: 0.166341, bitrate: 381 kb/s
  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 378 kb/s, 24 fps, 24 tbr, 12288 tbn, 48 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]

Initial segment produced by ffmpeg 4.4.1-static (Duration: N/A):

ffprobe version n4.4 Copyright (c) 2007-2021 the FFmpeg developers
  built with gcc 11.1.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x560481b4f240] Could not find codec parameters for stream 0 (Video: h264 (avc1 / 0x31637661), none, 1280x720): unspecified pixel format
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'init.mp4':
  Metadata:
    major_brand     : iso5
    minor_version   : 512
    compatible_brands: iso5iso6mp41
    encoder         : Lavf58.76.100
  Duration: N/A, bitrate: N/A
  Stream #0:0(und): Video: h264 (avc1 / 0x31637661), none, 1280x720, SAR 1:1 DAR 16:9, 12288 tbr, 12288 tbn, 24576 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]

Change History (1)

comment:1 by Steven Liu, 3 months ago

The option named hls_init_time is not be used set init.mp4 duration, it is used for hls live streaming first window list size segments duration time.
the init.mp4 is only storage mp4 header, will not storage video or audio data.
you can read init.mp4 first to build mp4 header and read fragments to on file.

./ffmpeg -f lavfi -i testsrc2=s=176x144:d=60:r=12 -g 24 -sc_threshold 0 -f hls -hls_time 6 -hls_segment_type fmp4 -hls_init_time 0 out.m3u8

split fragment 6s per fragment.
there should 5 fragments in out.m3u8

(base) liuqideMacBook-Pro:build liuqi$ cat out.m3u8
#EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:5
#EXT-X-MAP:URI="init.mp4"
#EXTINF:6.000000,
out5.m4s
#EXTINF:6.000000,
out6.m4s
#EXTINF:6.000000,
out7.m4s
#EXTINF:6.000000,
out8.m4s
#EXTINF:6.000000,
out9.m4s
#EXT-X-ENDLIST
(base) liuqideMacBook-Pro:build liuqi$
cat init.mp4 out2.m4s out3.m4s | ffplay -

This command should play from 12s.

the sequence from 0.

(base) liuqideMacBook-Pro:build liuqi$ ./ffmpeg -f lavfi -i testsrc2=s=176x144:d=60:r=12 -g 24 -sc_threshold 0 -f hls -hls_time 6 -hls_segment_type fmp4 -hls_init_time 0 -hls_list_size 0 out.m3u8
ffmpeg version N-104454-gd92fdc7144 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
  configuration: --enable-fontconfig --enable-gpl --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libspeex --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-version3 --enable-nonfree --enable-videotoolbox --enable-libxml2 --samples=/Users/liuqi/multimedia/fate-suite/ --enable-libopencv
  libavutil      57.  7.100 / 57.  7.100
  libavcodec     59. 12.100 / 59. 12.100
  libavformat    59.  8.100 / 59.  8.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8. 16.100 /  8. 16.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Input #0, lavfi, from 'testsrc2=s=176x144:d=60:r=12':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 176x144 [SAR 1:1 DAR 11:9], 12 tbr, 12 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x7fb6335157c0] using SAR=1/1
[libx264 @ 0x7fb6335157c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x7fb6335157c0] profile High, level 1.0
[libx264 @ 0x7fb6335157c0] 264 - core 148 r2694 3b70645 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=4 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=24 keyint_min=2 scenecut=0 intra_refresh=0 rc_lookahead=24 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[hls @ 0x7fb633514b80] Opening 'init.mp4' for writing
Output #0, hls, to 'out.m3u8':
  Metadata:
    encoder         : Lavf59.8.100
  Stream #0:0: Video: h264, yuv420p(progressive), 176x144 [SAR 1:1 DAR 11:9], q=2-31, 12 fps, 12288 tbn
    Metadata:
      encoder         : Lavc59.12.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
[hls @ 0x7fb633514b80] Opening 'out0.m4s' for writingitrate=N/A speed=   0x
[hls @ 0x7fb633514b80] Opening 'out.m3u8.tmp' for writing
[hls @ 0x7fb633514b80] Opening 'out1.m4s' for writing
[hls @ 0x7fb633514b80] Opening 'out.m3u8.tmp' for writing
[hls @ 0x7fb633514b80] Opening 'out2.m4s' for writing
[hls @ 0x7fb633514b80] Opening 'out.m3u8.tmp' for writing
[hls @ 0x7fb633514b80] Opening 'out3.m4s' for writing
[hls @ 0x7fb633514b80] Opening 'out.m3u8.tmp' for writing
[hls @ 0x7fb633514b80] Opening 'out4.m4s' for writing
[hls @ 0x7fb633514b80] Opening 'out.m3u8.tmp' for writing
[hls @ 0x7fb633514b80] Opening 'out5.m4s' for writingbitrate=N/A speed=68.8x
[hls @ 0x7fb633514b80] Opening 'out.m3u8.tmp' for writing
[hls @ 0x7fb633514b80] Opening 'out6.m4s' for writing
[hls @ 0x7fb633514b80] Opening 'out.m3u8.tmp' for writing
[hls @ 0x7fb633514b80] Opening 'out7.m4s' for writing
[hls @ 0x7fb633514b80] Opening 'out.m3u8.tmp' for writing
[hls @ 0x7fb633514b80] Opening 'out8.m4s' for writing
[hls @ 0x7fb633514b80] Opening 'out.m3u8.tmp' for writing
[hls @ 0x7fb633514b80] Opening 'out9.m4s' for writing
[hls @ 0x7fb633514b80] Opening 'out.m3u8.tmp' for writing
frame=  720 fps=0.0 q=-1.0 Lsize=N/A time=00:00:59.83 bitrate=N/A speed=68.9x
video:875kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0x7fb6335157c0] frame I:30    Avg QP:18.06  size:  4103
[libx264 @ 0x7fb6335157c0] frame P:209   Avg QP:25.75  size:  1640
[libx264 @ 0x7fb6335157c0] frame B:481   Avg QP:29.38  size:   893
[libx264 @ 0x7fb6335157c0] consecutive B-frames:  6.1%  7.2% 21.7% 65.0%
[libx264 @ 0x7fb6335157c0] mb I  I16..4: 36.2% 11.1% 52.7%
[libx264 @ 0x7fb6335157c0] mb P  I16..4:  4.7%  4.1%  5.1%  P16..4: 19.8% 15.2% 12.3%  0.0%  0.0%    skip:38.7%
[libx264 @ 0x7fb6335157c0] mb B  I16..4:  0.8%  0.3%  0.6%  B16..8: 27.6% 15.3%  7.0%  direct: 4.1%  skip:44.2%  L0:48.1% L1:44.6% BI: 7.3%
[libx264 @ 0x7fb6335157c0] 8x8 transform intra:19.8% inter:20.7%
[libx264 @ 0x7fb6335157c0] coded y,uvDC,uvAC intra: 28.7% 58.7% 50.6% inter: 13.2% 31.2% 27.1%
[libx264 @ 0x7fb6335157c0] i16 v,h,dc,p: 79% 15%  6%  0%
[libx264 @ 0x7fb6335157c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16%  7% 76%  0%  0%  0%  0%  0%  0%
[libx264 @ 0x7fb6335157c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 48% 24% 21%  1%  1%  1%  2%  2%  2%
[libx264 @ 0x7fb6335157c0] i8c dc,h,v,p: 42% 19% 37%  3%
[libx264 @ 0x7fb6335157c0] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x7fb6335157c0] ref P L0: 58.9%  7.8% 17.0% 16.2%
[libx264 @ 0x7fb6335157c0] ref B L0: 80.9% 15.4%  3.7%
[libx264 @ 0x7fb6335157c0] ref B L1: 93.1%  6.9%
[libx264 @ 0x7fb6335157c0] kb/s:119.36
(base) liuqideMacBook-Pro:build liuqi$

it's ok, or am i misunderstand some things?

Note: See TracTickets for help on using tickets.