Opened 6 years ago

Closed 6 years ago

#6776 closed defect (fixed)

hls muxer does not include the first segment in m3u8 files when segmenting to fmp4

Reported by: pickles Owned by: Steven Liu
Priority: normal Component: undetermined
Version: git-master Keywords: hls
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

The hls muxer does not include the first segment in the m3u8 file when hls_segment_type is set to output fmp4 segments, so playback only begins from the second segment onwards. Setting -hls_list_size 0 does not help.

How to reproduce:

ffmpeg -i input.mp4 -hls_segment_filename '%d.m4s' -hls_segment_type fmp4 -f hls fmp4.m3u8
ffmpeg version N-88116-gacf7063 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 4.9.2 (Debian 4.9.2-10)
  configuration: --prefix=/usr --enable-avfilter --enable-shared --enable-avresample --enable-gpl --enable-hardcoded-tables --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-nonfree --enable-postproc --enable-pthreads --enable-shared --enable-version3 --extra-libs=-lgcrypt
  libavutil      56.  0.100 / 56.  0.100
  libavcodec     58.  0.100 / 58.  0.100
  libavformat    58.  0.100 / 58.  0.100
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  0.100 /  7.  0.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  0.100 /  5.  0.100
  libswresample   3.  0.100 /  3.  0.100
  libpostproc    55.  0.100 / 55.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf57.76.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 3528 kb/s
    Stream #0:0(eng): Video: mpeg4 (Simple Profile) (mp4v / 0x7634706D), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 3350 kb/s, 25 fps, 25 tbr, 12800 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 174 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg4 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0x15f6620] using SAR=1/1
[libx264 @ 0x15f6620] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
[libx264 @ 0x15f6620] profile High, level 3.0
[libx264 @ 0x15f6620] 264 - core 142 r2431 a5831aa - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[hls @ 0x15f6040] Opening 'init.mp4' for writing
Output #0, hls, to 'fmp4.m3u8':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf58.0.100
    Stream #0:0(eng): Video: h264 (libx264), yuv420p, 640x480 [SAR 1:1 DAR 4:3], q=-1--1, 25 fps, 12800 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc58.0.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1(eng): Audio: aac (LC), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc58.0.100 aac
[hls @ 0x15f6040] Opening '0.m4s' for writing
[hls @ 0x15f6040] Opening 'fmp4.m3u8.tmp' for writing
[mp4 @ 0x1d9a4a0] Track 1 starts with a nonzero dts 5723, while the moov already has been written. Set the delay_moov flag to handle this case.
[hls @ 0x15f6040] Opening '1.m4s' for writing0:07.94 bitrate=N/A speed=2.22x    
[hls @ 0x15f6040] Opening 'fmp4.m3u8.tmp' for writing
[hls @ 0x15f6040] Opening '2.m4s' for writing0:09.80 bitrate=N/A speed=2.13x    
[hls @ 0x15f6040] Opening 'fmp4.m3u8.tmp' for writing
[hls @ 0x15f6040] Opening '3.m4s' for writing
[hls @ 0x15f6040] Opening 'fmp4.m3u8.tmp' for writing
[hls @ 0x15f6040] Opening 'fmp4.m3u8.tmp' for writing
frame=  250 fps= 45 q=-1.0 Lsize=N/A time=00:00:10.00 bitrate=N/A speed= 1.8x    
video:1229kB audio:115kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

This creates 0.m4s, 1.m4s, 2.m4s and 3.m4s but the m3u8 does not include 0.m4s. The resulting file therefore misses the first segment:

#EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:7
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-MAP:URI="init.mp4"
#EXTINF:1.520000,
1.m4s
#EXTINF:1.280000,
2.m4s
#EXTINF:0.320000,
3.m4s
#EXT-X-ENDLIST

Using the same ffmpeg command without the -hls_segment_type fmp4 option creates mpegts segments (0.ts, 1.ts, 2.ts and 3.ts) and the corresponding m3u8 includes all segments, so nothing is skipped:

ffmpeg -i input.mp4 -hls_segment_filename '%d.ts' -f hls mpegts.m3u8
ffmpeg version N-88116-gacf7063 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 4.9.2 (Debian 4.9.2-10)
  configuration: --prefix=/usr --enable-avfilter --enable-shared --enable-avresample --enable-gpl --enable-hardcoded-tables --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-nonfree --enable-postproc --enable-pthreads --enable-shared --enable-version3 --extra-libs=-lgcrypt
  libavutil      56.  0.100 / 56.  0.100
  libavcodec     58.  0.100 / 58.  0.100
  libavformat    58.  0.100 / 58.  0.100
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  0.100 /  7.  0.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  0.100 /  5.  0.100
  libswresample   3.  0.100 /  3.  0.100
  libpostproc    55.  0.100 / 55.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf57.76.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 3528 kb/s
    Stream #0:0(eng): Video: mpeg4 (Simple Profile) (mp4v / 0x7634706D), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 3350 kb/s, 25 fps, 25 tbr, 12800 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 174 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg4 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0x9dc5c0] using SAR=1/1
[libx264 @ 0x9dc5c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
[libx264 @ 0x9dc5c0] profile High, level 3.0
[libx264 @ 0x9dc5c0] 264 - core 142 r2431 a5831aa - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[hls @ 0x9dbfe0] Opening '0.ts' for writing
Output #0, hls, to 'mpegts.m3u8':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf58.0.100
    Stream #0:0(eng): Video: h264 (libx264), yuv420p, 640x480 [SAR 1:1 DAR 4:3], q=-1--1, 25 fps, 90k tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc58.0.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1(eng): Audio: aac (LC), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc58.0.100 aac
[hls @ 0x9dbfe0] Opening '1.ts' for writing:00:08.75 bitrate=N/A speed=2.14x    
[hls @ 0x9dbfe0] Opening 'mpegts.m3u8.tmp' for writing
[hls @ 0x9dbfe0] Opening '2.ts' for writing:00:09.75 bitrate=N/A speed=2.12x    
[hls @ 0x9dbfe0] Opening 'mpegts.m3u8.tmp' for writing
[hls @ 0x9dbfe0] Opening '3.ts' for writing
[hls @ 0x9dbfe0] Opening 'mpegts.m3u8.tmp' for writing
[hls @ 0x9dbfe0] Opening 'mpegts.m3u8.tmp' for writing
frame=  250 fps= 45 q=-1.0 Lsize=N/A time=00:00:10.00 bitrate=N/A speed=1.79x    
video:1229kB audio:115kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

resulting m3u8:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:7
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:6.880000,
0.ts
#EXTINF:1.520000,
1.ts
#EXTINF:1.280000,
2.ts
#EXTINF:0.320000,
3.ts
#EXT-X-ENDLIST

In this particular example, playing the fmp4 segmented version means the first 7 seconds are missed, whereas the mpegts segmented video is complete.

Change History (11)

comment:1 by Carl Eugen Hoyos, 6 years ago

Is specific input needed to reproduce or is the issue also reproducible with -f lavfi -i testsrc?

in reply to:  1 comment:2 by pickles, 6 years ago

Replying to cehoyos:

Is specific input needed to reproduce or is the issue also reproducible with -f lavfi -i testsrc?

It is still reproducible with -f lavfi -i testsrc:

ffmpeg -f lavfi -i testsrc -hls_segment_filename '%d.m4s' -hls_segment_type fmp4 -hls_fmp4_init_filename init.mp4 -f hls -t 30 fmp4.m3u8
ffmpeg version N-88116-gacf70639fb Copyright (c) 2000-2017 the FFmpeg developers
  built with Apple LLVM version 9.0.0 (clang-900.0.38)
  configuration: --prefix=/usr/local --enable-avfilter --enable-static --enable-pthreads --enable-gpl --enable-version3 --enable-nonfree --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libmp3lame --enable-libxvid --enable-ffplay --enable-libfdk-aac
  libavutil      56.  0.100 / 56.  0.100
  libavcodec     58.  0.100 / 58.  0.100
  libavformat    58.  0.100 / 58.  0.100
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  0.100 /  7.  0.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  0.100 /  5.  0.100
  libswresample   3.  0.100 /  3.  0.100
  libpostproc    55.  0.100 / 55.  0.100
Input #0, lavfi, from 'testsrc':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 320x240 [SAR 1:1 DAR 4:3], 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x7fcdf6002a00] using SAR=1/1
[libx264 @ 0x7fcdf6002a00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 0x7fcdf6002a00] profile High 4:4:4 Predictive, level 1.3, 4:4:4 8-bit
[libx264 @ 0x7fcdf6002a00] 264 - core 148 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[hls @ 0x7fcdf6801600] Opening 'init.mp4' for writing
Output #0, hls, to 'fmp4.m3u8':
  Metadata:
    encoder         : Lavf58.0.100
    Stream #0:0: Video: h264 (libx264), yuv444p, 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 25 fps, 12800 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.0.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
[hls @ 0x7fcdf6801600] Opening '0.m4s' for writing
[hls @ 0x7fcdf6801600] Opening 'fmp4.m3u8.tmp' for writing
[hls @ 0x7fcdf6801600] Opening '1.m4s' for writing
[hls @ 0x7fcdf6801600] Opening 'fmp4.m3u8.tmp' for writing
[hls @ 0x7fcdf6801600] Opening '2.m4s' for writing80 bitrate=N/A speed=29.5x    
[hls @ 0x7fcdf6801600] Opening 'fmp4.m3u8.tmp' for writing
[hls @ 0x7fcdf6801600] Opening 'fmp4.m3u8.tmp' for writing
frame=  750 fps=0.0 q=-1.0 Lsize=N/A time=00:00:29.88 bitrate=N/A speed=34.3x    
video:130kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

The command should have created a 30 second video but it only plays the last 20 seconds as the first segment (0.m4s) is omitted from the m3u8 file:

#EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-MAP:URI="init.mp4"
#EXTINF:10.000000,
1.m4s
#EXTINF:10.000000,
2.m4s
#EXT-X-ENDLIST

comment:3 by Carl Eugen Hoyos, 6 years ago

Keywords: hls added
Version: unspecifiedgit-master

A similar command line writes a file with a corrupt filename and corrupt content to the filesystem here, does this not happen to you?

in reply to:  3 ; comment:4 by pickles, 6 years ago

Replying to cehoyos:

A similar command line writes a file with a corrupt filename and corrupt content to the filesystem here, does this not happen to you?

Not on Debian Linux but it does on macOS although I find if I keep retrying the same command then eventually it works. I had assumed this was an incompatibility with the new macOS filesystem which changed in High Sierra because it always worked fine in Linux. I can, however, rename the corrupt filename to 'init.mp4' manually and then play the .m3u8 file via ffplay.

in reply to:  4 comment:5 by pickles, 6 years ago

Replying to pickles:

Replying to cehoyos:

A similar command line writes a file with a corrupt filename and corrupt content to the filesystem here, does this not happen to you?

Not on Debian Linux but it does on macOS although I find if I keep retrying the same command then eventually it works. I had assumed this was an incompatibility with the new macOS filesystem which changed in High Sierra because it always worked fine in Linux. I can, however, rename the corrupt filename to 'init.mp4' manually and then play the .m3u8 file via ffplay.

Further testing on Debian Linux reveals the same problem. The reason I did not notice before is because I was providing an absolute path to -hls_fmp4_init_filename whereas in macOS I wasn't. Without an absolute path, the problem does indeed also occur in Linux. So this must be another (unrelated) bug.

ffmpeg -f lavfi -i testsrc -hls_segment_filename '/tmp/%d.m4s' -hls_segment_type fmp4 -hls_fmp4_init_filename /tmp/init.mp4 -f hls -t 30 /tmp/fmp4.m3u8

creates the files without filesystem corruption. Of course the main issue still remains, in that the m3u8 file does not include the first segment.

comment:6 by pickles, 6 years ago

I have created a new ticket regarding the filesystem corruption: #6777


I have used git bisect to ascertain the regression for the issue of the m3u8 not including the first segment:

738b29cfb69b82965fd71d00da85f162b04a171e is the first bad commit

comment:7 by Steven Liu, 6 years ago

Owner: set to Steven Liu
Status: newopen

let me fix it

comment:8 by Steven Liu, 6 years ago

comment:9 by Steven Liu, 6 years ago

comment:10 by pickles, 6 years ago

I can confirm that the patch works and fixes the problem. Thanks!

comment:11 by Steven Liu, 6 years ago

Resolution: fixed
Status: openclosed
Note: See TracTickets for help on using tickets.