Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#7082 closed defect (invalid)

HLS: m3u8 file is missing the first segment

Reported by: Ștefan Talpalaru Owned by:
Priority: normal Component: undetermined
Version: unspecified Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug: a certain mp4 input file results in a m3u8 mapping with a missing first segment

How to reproduce (HLSv4):

% ffmpeg -i https://senatus.net/media/files/senatus_5GNqzN.mp4 -hls_time 10 -hls_list_size 10 -hls_flags single_file -hls_segment_filename senatus_5GNqzN_orig.ts -c copy out.m3u8

The resulting out.m3u8 is missing the first segment:

#EXTM3U
#EXT-X-VERSION:4
#EXT-X-TARGETDURATION:16
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:11.644967,
#EXT-X-BYTERANGE:2399444@1591984
senatus_5GNqzN_orig.ts
#EXTINF:10.477133,
#EXT-X-BYTERANGE:2011036@3991428
senatus_5GNqzN_orig.ts
#EXTINF:8.141467,
#EXT-X-BYTERANGE:1340628@6002464
senatus_5GNqzN_orig.ts
#EXTINF:10.643967,
#EXT-X-BYTERANGE:1986596@7343092
senatus_5GNqzN_orig.ts
#EXTINF:9.376033,
#EXT-X-BYTERANGE:1459820@9329688
senatus_5GNqzN_orig.ts
#EXTINF:10.176833,
#EXT-X-BYTERANGE:1259224@10789508
senatus_5GNqzN_orig.ts
#EXTINF:10.176833,
#EXT-X-BYTERANGE:1676772@12048732
senatus_5GNqzN_orig.ts
#EXTINF:15.982633,
#EXT-X-BYTERANGE:2827520@13725504
senatus_5GNqzN_orig.ts
#EXTINF:4.604600,
#EXT-X-BYTERANGE:201912@16553024
senatus_5GNqzN_orig.ts
#EXTINF:4.938967,
#EXT-X-BYTERANGE:461916@16754936
senatus_5GNqzN_orig.ts
#EXT-X-ENDLIST

An HLSv3 file also misses the first 10s segment:

ffmpeg -y -i https://senatus.net/media/files/senatus_5GNqzN.mp4 -hls_time 10 -hls_list_size 10 -c copy out2.m3u8

results in:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:16
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:11.644967,
out21.ts
#EXTINF:10.477133,
out22.ts
#EXTINF:8.141467,
out23.ts
#EXTINF:10.643967,
out24.ts
#EXTINF:9.376033,
out25.ts
#EXTINF:10.176833,
out26.ts
#EXTINF:10.176833,
out27.ts
#EXTINF:15.982633,
out28.ts
#EXTINF:4.604600,
out29.ts
#EXTINF:4.938967,
out210.ts
#EXT-X-ENDLIST

(there should be an "out20.ts" in the first position and the ts file is generated, just not added to the m3u8 file)

There are no errors in the ffmpeg output. Here's the one for the first command:

ffmpeg version 3.4.2 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7.3.0 (Gentoo 7.3.0 p1.0)
  configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/share/doc/ffmpeg-3.4.2/html --mandir=/usr/share/man --enable-shared --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --optflags='-O3 -march=native -pipe' --disable-static --enable-avfilter --enable-avresample --disable-stripping --enable-nonfree --disable-indev=v4l2 --disable-outdev=v4l2 --disable-indev=oss --disable-outdev=oss --enable-nonfree --enable-bzlib --disable-runtime-cpudetect --disable-debug --disable-gcrypt --disable-gnutls --disable-gmp --enable-gpl --enable-hardcoded-tables --enable-iconv --enable-lzma --enable-network --enable-openssl --enable-postproc --disable-libsmbclient --enable-ffplay --enable-sdl2 --enable-vaapi --enable-vdpau --enable-xlib --enable-libxcb --enable-libxcb-shm --enable-libxcb-xfixes --enable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --disable-libcaca --disable-openal --enable-opengl --disable-libv4l2 --disable-libpulse --disable-libdrm --disable-libopencore-amrwb --disable-libopencore-amrnb --enable-libfdk-aac --disable-libopenjpeg --disable-libbluray --disable-libcelt --disable-libgme --disable-libgsm --disable-mmal --disable-libmodplug --enable-libopus --disable-libilbc --disable-librtmp --disable-libssh --enable-libspeex --enable-librsvg --enable-libvorbis --enable-libvpx --enable-libzvbi --disable-appkit --disable-libbs2b --disable-chromaprint --disable-libflite --disable-frei0r --disable-libfribidi --enable-fontconfig --enable-ladspa --enable-libass --enable-libfreetype --disable-librubberband --disable-libzmq --disable-libzimg --disable-libsoxr --enable-pthreads --disable-libvo-amrwbenc --enable-libmp3lame --disable-libkvazaar --disable-nvenc --disable-libopenh264 --disable-libsnappy --enable-libtheora --disable-libtwolame --disable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --disable-armv5te --disable-armv6 --disable-armv6t2 --disable-neon --disable-vfp --disable-vfpv3 --disable-armv8 --disable-mipsdsp --disable-mipsdspr2 --disable-mipsfpu --disable-altivec --disable-amd3dnow --disable-amd3dnowext --disable-avx2 --cpu=host --disable-doc --disable-htmlpages --enable-manpages
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'https://senatus.net/media/files/senatus_5GNqzN.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.83.100
  Duration: 00:01:46.46, start: 0.000000, bitrate: 1235 kb/s
    Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 1101 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
[hls @ 0x559156248d40] Opening 'senatus_5GNqzN_orig.ts' for writing
Output #0, hls, to 'out.m3u8':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.83.100
    Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 1101 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 29.97 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[hls @ 0x559156248d40] Opening 'out.m3u8.tmp' for writingate=N/A speed=4.54x    
[hls @ 0x559156248d40] Opening 'out.m3u8.tmp' for writingate=N/A speed=4.19x    
[hls @ 0x559156248d40] Opening 'out.m3u8.tmp' for writingate=N/A speed= 4.2x    
[hls @ 0x559156248d40] Opening 'out.m3u8.tmp' for writingate=N/A speed=4.52x    
[hls @ 0x559156248d40] Opening 'out.m3u8.tmp' for writingate=N/A speed=4.94x    
[hls @ 0x559156248d40] Opening 'out.m3u8.tmp' for writingate=N/A speed=5.29x    
[hls @ 0x559156248d40] Opening 'out.m3u8.tmp' for writingate=N/A speed=5.29x    
[hls @ 0x559156248d40] Opening 'out.m3u8.tmp' for writingate=N/A speed= 5.9x    
[hls @ 0x559156248d40] Opening 'out.m3u8.tmp' for writingate=N/A speed=5.76x    
    Last message repeated 1 times
[hls @ 0x559156248d40] Opening 'out.m3u8.tmp' for writingate=N/A speed=6.27x    
frame= 3189 fps=193 q=-1.0 Lsize=N/A time=00:01:46.39 bitrate=N/A speed=6.45x    
video:14306kB audio:1666kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

Change History (6)

comment:1 by Steven Liu, 6 years ago

-hls_list_size 0 will ok, test bellow:

liuqideMacBook-Pro:xxx liuqi$ ./ffmpeg -i ./senatus_5GNqzN.mp4 -hls_time 10 -hls_list_size 10 -hls_flags single_file -hls_segment_filename senatus_5GNqzN_orig.ts -c copy out.m3u8
ffmpeg version N-90246-g31bd104cad Copyright (c) 2000-2018 the FFmpeg developers
  built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
  configuration: --enable-fontconfig --enable-gpl --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libspeex --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-version3 --cc='ccache gcc' --enable-nonfree --enable-videotoolbox --enable-opengl
  libavutil      56.  8.100 / 56.  8.100
  libavcodec     58. 13.102 / 58. 13.102
  libavformat    58. 10.100 / 58. 10.100
  libavdevice    58.  2.100 / 58.  2.100
  libavfilter     7. 12.100 /  7. 12.100
  libswscale      5.  0.102 /  5.  0.102
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './senatus_5GNqzN.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.83.100
  Duration: 00:01:46.46, start: 0.000000, bitrate: 1235 kb/s
    Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 1101 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
[hls @ 0x7fc7c6800a00] Opening 'senatus_5GNqzN_orig.ts' for writing
Output #0, hls, to 'out.m3u8':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.10.100
    Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 1101 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 29.97 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[hls @ 0x7fc7c6800a00] Opening 'out.m3u8.tmp' for writing
    Last message repeated 9 times
[hls @ 0x7fc7c6800a00] Opening 'out.m3u8.tmp' for writing
frame= 3189 fps=0.0 q=-1.0 Lsize=N/A time=00:01:46.39 bitrate=N/A speed= 543x
video:14306kB audio:1666kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
liuqideMacBook-Pro:xxx liuqi$ cat out.m3u8
#EXTM3U
#EXT-X-VERSION:4
#EXT-X-TARGETDURATION:16
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:11.644967,
#EXT-X-BYTERANGE:2399444@1591984
senatus_5GNqzN_orig.ts
#EXTINF:10.477133,
#EXT-X-BYTERANGE:2011036@3991428
senatus_5GNqzN_orig.ts
#EXTINF:8.141467,
#EXT-X-BYTERANGE:1340628@6002464
senatus_5GNqzN_orig.ts
#EXTINF:10.643967,
#EXT-X-BYTERANGE:1986596@7343092
senatus_5GNqzN_orig.ts
#EXTINF:9.376033,
#EXT-X-BYTERANGE:1459820@9329688
senatus_5GNqzN_orig.ts
#EXTINF:10.176833,
#EXT-X-BYTERANGE:1259224@10789508
senatus_5GNqzN_orig.ts
#EXTINF:10.176833,
#EXT-X-BYTERANGE:1676772@12048732
senatus_5GNqzN_orig.ts
#EXTINF:15.982633,
#EXT-X-BYTERANGE:2827520@13725504
senatus_5GNqzN_orig.ts
#EXTINF:4.604600,
#EXT-X-BYTERANGE:201912@16553024
senatus_5GNqzN_orig.ts
#EXTINF:4.938967,
#EXT-X-BYTERANGE:461916@16754936
senatus_5GNqzN_orig.ts
#EXT-X-ENDLIST
liuqideMacBook-Pro:xxx liuqi$
liuqideMacBook-Pro:xxx liuqi$
liuqideMacBook-Pro:xxx liuqi$
liuqideMacBook-Pro:xxx liuqi$
liuqideMacBook-Pro:xxx liuqi$
liuqideMacBook-Pro:xxx liuqi$
liuqideMacBook-Pro:xxx liuqi$
liuqideMacBook-Pro:xxx liuqi$
liuqideMacBook-Pro:xxx liuqi$
liuqideMacBook-Pro:xxx liuqi$ ./ffmpeg -i ./senatus_5GNqzN.mp4 -hls_time 10 -hls_list_size 0 -hls_flags single_file -hls_segment_filename senatus_5GNqzN_orig.ts -c copy out.m3u8
ffmpeg version N-90246-g31bd104cad Copyright (c) 2000-2018 the FFmpeg developers
  built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
  configuration: --enable-fontconfig --enable-gpl --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libspeex --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-version3 --cc='ccache gcc' --enable-nonfree --enable-videotoolbox --enable-opengl
  libavutil      56.  8.100 / 56.  8.100
  libavcodec     58. 13.102 / 58. 13.102
  libavformat    58. 10.100 / 58. 10.100
  libavdevice    58.  2.100 / 58.  2.100
  libavfilter     7. 12.100 /  7. 12.100
  libswscale      5.  0.102 /  5.  0.102
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './senatus_5GNqzN.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.83.100
  Duration: 00:01:46.46, start: 0.000000, bitrate: 1235 kb/s
    Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 1101 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
[hls @ 0x7f867a002600] Opening 'senatus_5GNqzN_orig.ts' for writing
Output #0, hls, to 'out.m3u8':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.10.100
    Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 1101 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 29.97 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[hls @ 0x7f867a002600] Opening 'out.m3u8.tmp' for writing
    Last message repeated 9 times
[hls @ 0x7f867a002600] Opening 'out.m3u8.tmp' for writing
frame= 3189 fps=0.0 q=-1.0 Lsize=N/A time=00:01:46.39 bitrate=N/A speed= 586x
video:14306kB audio:1666kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
liuqideMacBook-Pro:xxx liuqi$
liuqideMacBook-Pro:xxx liuqi$
liuqideMacBook-Pro:xxx liuqi$ cat out.m3u8
#EXTM3U
#EXT-X-VERSION:4
#EXT-X-TARGETDURATION:16
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:10.243567,
#EXT-X-BYTERANGE:1591984@0
senatus_5GNqzN_orig.ts
#EXTINF:11.644967,
#EXT-X-BYTERANGE:2399444@1591984
senatus_5GNqzN_orig.ts
#EXTINF:10.477133,
#EXT-X-BYTERANGE:2011036@3991428
senatus_5GNqzN_orig.ts
#EXTINF:8.141467,
#EXT-X-BYTERANGE:1340628@6002464
senatus_5GNqzN_orig.ts
#EXTINF:10.643967,
#EXT-X-BYTERANGE:1986596@7343092
senatus_5GNqzN_orig.ts
#EXTINF:9.376033,
#EXT-X-BYTERANGE:1459820@9329688
senatus_5GNqzN_orig.ts
#EXTINF:10.176833,
#EXT-X-BYTERANGE:1259224@10789508
senatus_5GNqzN_orig.ts
#EXTINF:10.176833,
#EXT-X-BYTERANGE:1676772@12048732
senatus_5GNqzN_orig.ts
#EXTINF:15.982633,
#EXT-X-BYTERANGE:2827520@13725504
senatus_5GNqzN_orig.ts
#EXTINF:4.604600,
#EXT-X-BYTERANGE:201912@16553024
senatus_5GNqzN_orig.ts
#EXTINF:4.938967,
#EXT-X-BYTERANGE:461916@16754936
senatus_5GNqzN_orig.ts
#EXT-X-ENDLIST
liuqideMacBook-Pro:xxx liuqi$

comment:2 by Ștefan Talpalaru, 6 years ago

Thank you, setting "hls_list_size" to 0 fixes my problem.

Can 0 be made the default value instead of 5? It doesn't really make sense to not generate all the HLS segments by default for my use case (not a live stream, but a fixed-size video with different resolutions served through HLS to allow the in-browser player to auto-select the appropriate one for the client's bandwidth).

in reply to:  2 ; comment:3 by Steven Liu, 6 years ago

Resolution: fixed
Status: newclosed

Replying to stefantalpalaru:

Thank you, setting "hls_list_size" to 0 fixes my problem.

Can 0 be made the default value instead of 5? It doesn't really make sense to not generate all the HLS segments by default for my use case (not a live stream, but a fixed-size video with different resolutions served through HLS to allow the in-browser player to auto-select the appropriate one for the client's bandwidth).

default is living stream mode, one m3u8 have 5 fragments, so that have no problem, you can use hls_list_size to set it :)

in reply to:  3 ; comment:4 by Karthick Jeyapal, 6 years ago

Replying to stevenliu:

Replying to stefantalpalaru:

Thank you, setting "hls_list_size" to 0 fixes my problem.

Can 0 be made the default value instead of 5? It doesn't really make sense to not generate all the HLS segments by default for my use case (not a live stream, but a fixed-size video with different resolutions served through HLS to allow the in-browser player to auto-select the appropriate one for the client's bandwidth).

default is living stream mode, one m3u8 have 5 fragments, so that have no problem, you can use hls_list_size to set it :)

Does this mean, we will always lose the first segment during live streaming? If yes, then this still sounds like a bug.

in reply to:  4 comment:5 by Steven Liu, 6 years ago

Replying to j_karthic:

Replying to stevenliu:

Replying to stefantalpalaru:

Thank you, setting "hls_list_size" to 0 fixes my problem.

Can 0 be made the default value instead of 5? It doesn't really make sense to not generate all the HLS segments by default for my use case (not a live stream, but a fixed-size video with different resolutions served through HLS to allow the in-browser player to auto-select the appropriate one for the client's bandwidth).

default is living stream mode, one m3u8 have 5 fragments, so that have no problem, you can use hls_list_size to set it :)

Does this mean, we will always lose the first segment during live streaming? If yes, then this still sounds like a bug.

No, hls_list_size 0 can fix this problem, you set hls_list_size to 5, then there have 6 fragments, it will remove the frist fragment to keep the m3u8 list window size is 5 fragments. What about you think?

Last edited 6 years ago by Steven Liu (previous) (diff)

comment:6 by Carl Eugen Hoyos, 6 years ago

Resolution: fixedinvalid
Version: 3.4unspecified
Note: See TracTickets for help on using tickets.