Seeking to beginning of HLS stream fails

[tospi]

I created an HLS stream with the following command:

E:\ffmpeg.exe -lavfi testsrc2=duration=30 hls/file.m3u8

I played the stream with the following command:

E:\ffplay.exe hls/file.m3u8

When I try to seek to the beginning (left arrow key pressed once during the first seconds), then the playback jumps to the beginning of the second segment (00:00:10) instead of the beginning of the stream (00:00:00). In addition, the playback speed is somehow slower then normal after this seek.

My assumption is that the start offset is somehow wrong and this is causing the seek to fail. I am not sure if this is a problem during stream creation (muxer) or only on the playback side (demuxer?). I assume a problem on the playback side, because the seek is working in VLC 2.2.6.

I also tried opening the HLS stream via a webserver, but there is no difference:

E:\ffplay.exe http://192.168.0.101:8085/hls/file.m3u8

Output of stream creation:

% ffmpeg.exe -lavfi testsrc2=duration=30 hls/file.m3u8 -v verbose
ffmpeg version N-89127-g8f4702a93f Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      56.  0.100 / 56.  0.100
  libavcodec     58.  3.103 / 58.  3.103
  libavformat    58.  2.100 / 58.  2.100
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  2.100 /  7.  2.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
[Parsed_testsrc2_0 @ 00000267df8aa760] size:320x240 rate:25/1 duration:30.000000 sar:1/1
Stream mapping:
  testsrc2 -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
[Parsed_testsrc2_0 @ 00000267df8aaa40] size:320x240 rate:25/1 duration:30.000000 sar:1/1
[libx264 @ 00000267df8ad640] using SAR=1/1
[libx264 @ 00000267df8ad640] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 00000267df8ad640] profile High, level 1.3
[libx264 @ 00000267df8ad640] 264 - core 152 r2851 ba24899 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=7 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[hls @ 00000267df8ab380] Opening 'hls/file0.ts' for writing
[mpegts @ 00000267df974fe0] muxrate VBR, pcr every 2 pkts, sdt every 2147483647, pat/pmt every 2147483647 pkts
Output #0, hls, to 'hls/file.m3u8':
  Metadata:
    encoder         : Lavf58.2.100
    Stream #0:0: Video: h264 (libx264), 1 reference frame, yuv420p, 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 25 fps, 90k tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc58.3.103 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
[hls @ 00000267df8ab380] Opening 'hls/file1.ts' for writing
[hls @ 00000267df8ab380] Opening 'hls/file.m3u8.tmp' for writing
[hls muxer @ 00000267df8ab980] EXT-X-MEDIA-SEQUENCE:0
[hls @ 00000267df8ab380] Opening 'hls/file2.ts' for writing
[hls @ 00000267df8ab380] Opening 'hls/file.m3u8.tmp' for writing
[hls muxer @ 00000267df8ab980] EXT-X-MEDIA-SEQUENCE:0
[Parsed_testsrc2_0 @ 00000267df8aaa40] EOF timestamp not reliablespeed=41.7x
No more output streams to write to, finishing.
[hls @ 00000267df8ab380] Opening 'hls/file.m3u8.tmp' for writing
[hls muxer @ 00000267df8ab980] EXT-X-MEDIA-SEQUENCE:0
frame=  750 fps=0.0 q=-1.0 Lsize=N/A time=00:00:29.88 bitrate=N/A speed=43.2x
video:1019kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file #0 (hls/file.m3u8):
  Output stream #0:0 (video): 750 frames encoded; 750 packets muxed (1043850 bytes);
  Total: 750 packets (1043850 bytes) muxed
[libx264 @ 00000267df8ad640] frame I:3     Avg QP:18.79  size:  5454
[libx264 @ 00000267df8ad640] frame P:215   Avg QP:26.80  size:  2011
[libx264 @ 00000267df8ad640] frame B:532   Avg QP:31.45  size:  1117
[libx264 @ 00000267df8ad640] consecutive B-frames:  2.5%  5.9%  8.4% 83.2%
[libx264 @ 00000267df8ad640] mb I  I16..4: 35.6% 37.0% 27.4%
[libx264 @ 00000267df8ad640] mb P  I16..4:  2.4%  5.0%  1.2%  P16..4: 13.7% 10.2%  7.0%  0.0%  0.0%    skip:60.4%
[libx264 @ 00000267df8ad640] mb B  I16..4:  0.2%  0.3%  0.1%  B16..8: 19.9%  7.4%  2.0%  direct: 2.6%  skip:67.4%  L0:52.8% L1:41.7% BI: 5.5%
[libx264 @ 00000267df8ad640] 8x8 transform intra:54.3% inter:30.9%
[libx264 @ 00000267df8ad640] coded y,uvDC,uvAC intra: 10.1% 28.2% 23.9% inter: 5.4% 15.2% 12.6%
[libx264 @ 00000267df8ad640] i16 v,h,dc,p: 70% 25%  6%  0%
[libx264 @ 00000267df8ad640] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  6%  9% 84%  1%  0%  0%  0%  0%  0%
[libx264 @ 00000267df8ad640] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 40% 21% 34%  1%  1%  1%  1%  1%  1%
[libx264 @ 00000267df8ad640] i8c dc,h,v,p: 53% 17% 29%  1%
[libx264 @ 00000267df8ad640] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 00000267df8ad640] ref P L0: 58.3%  7.8% 20.0% 13.9%
[libx264 @ 00000267df8ad640] ref B L0: 79.7% 16.4%  3.9%
[libx264 @ 00000267df8ad640] ref B L1: 92.1%  7.9%
[libx264 @ 00000267df8ad640] kb/s:278.18

Output of playback:

% ffplay.exe hls/file.m3u8 -v verbose
ffplay version N-89127-g8f4702a93f Copyright (c) 2003-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      56.  0.100 / 56.  0.100
  libavcodec     58.  3.103 / 58.  3.103
  libavformat    58.  2.100 / 58.  2.100
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  2.100 /  7.  2.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Initialized direct3d renderer.
[hls,applehttp @ 000001f4de260060] HLS request for url 'hls/file0.ts', offset 0, playlist 0
[hls,applehttp @ 000001f4de260060] Opening 'hls/file0.ts' for reading
[h264 @ 000001f4de263be0] Reinit context to 320x240, pix_fmt: yuv420p
[hls,applehttp @ 000001f4de260060] max_analyze_duration 5000000 reached at 5000000 microseconds st:0
Input #0, hls,applehttp, from 'hls/file.m3u8':
  Duration: 00:00:30.00, start: 1.480000, bitrate: 0 kb/s
  Program 0
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Video: h264 (High), 1 reference frame ([27][0][0][0] / 0x001B), yuv420p(left), 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Metadata:
      variant_bitrate : 0
[h264 @ 000001f4e3627700] Reinit context to 320x240, pix_fmt: yuv420p
[ffplay_buffer @ 000001f4de277fe0] w:320 h:240 pixfmt:yuv420p tb:1/90000 fr:25/1 sar:1/1 sws_param:
Created 320x240 texture with SDL_PIXELFORMAT_IYUV.
[hls,applehttp @ 000001f4de260060] HLS request for url 'hls/file0.ts', offset 0, playlist 0
[hls,applehttp @ 000001f4de260060] Opening 'hls/file0.ts' for reading
[hls,applehttp @ 000001f4de260060] HLS request for url 'hls/file1.ts', offset 0, playlist 0
[hls,applehttp @ 000001f4de260060] Opening 'hls/file1.ts' for reading
[h264 @ 000001f4e3627700] Reinit context to 320x240, pix_fmt: yuv420p
[ffplay_buffer @ 000001f4de31ef20] w:320 h:240 pixfmt:yuv420p tb:1/90000 fr:25/1 sar:1/1 sws_param:
   4.15 M-V: -8.688 fd=   0 aq=    0KB vq=   34KB sq=    0B f=0/0
   5.90 M-V: -7.807 fd=   0 aq=    0KB vq=   35KB sq=    0B f=0/0

[Steven Liu]

ffmpeg -f lavfi -i testsrc2=duration=30 -vcodec mpeg2video -hls_time 10 hls/file.m3u8

use mpeg2video codec is ok.

[tospi]

Replying to stevenliu:

ffmpeg -f lavfi -i testsrc2=duration=30 -vcodec mpeg2video -hls_time 10 hls/file.m3u8

use mpeg2video codec is ok.

Mpeg2 is too old and has bad quality, so this is not really a solution...

With HEVC/H.265, the seeking problem is also the same by the way.

[Steven Liu]

Replying to tospi:

Replying to stevenliu:

ffmpeg -f lavfi -i testsrc2=duration=30 -vcodec mpeg2video -hls_time 10 hls/file.m3u8

use mpeg2video codec is ok.

Mpeg2 is too old and has bad quality, so this is not really a solution...

With HEVC/H.265, the seeking problem is also the same by the way.

Yes, i just want say, maybe need to process codec or mpegts part :D

[Steven Liu]

Mark process flag:
look at the phenomenon， looks like seek cannot seek to the first keyframe:
test command: ./ffmpeg -f lavfi -i testsrc2=duration=10 -r:v 25 -g 2 -hls_time 3 hls/file.m3u8
it seek to the second frame, 0.080s 40ms per frame. keyframe interval is 2

[Steven Liu]

Replying to tospi:

Replying to stevenliu:

ffmpeg -f lavfi -i testsrc2=duration=30 -vcodec mpeg2video -hls_time 10 hls/file.m3u8

use mpeg2video codec is ok.

Mpeg2 is too old and has bad quality, so this is not really a solution...

With HEVC/H.265, the seeking problem is also the same by the way.

try this patch please:
​https://patchwork.ffmpeg.org/patch/7076/

[Balling]

See ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210525125238.325548-1-gustav.grusell@gmail.com/ and ​​https://patchwork.ffmpeg.org/patch/7076/

Should fix this, second was NAK'ed but I suppose there may be some sence there too.

Also #7485 is a duplicate.

[Gustav Grusell]

As far as I can tell #7485 is not a duplicate even though it it very similar. Different types of seek is used in the two scenarios: In the scenario above, ffplay will use the seek_frame_byte funtion for seeking, while in the scenario of #7485 hls_read_seek is used.

Related to the above, the patch ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210525125238.325548-1-gustav.grusell@gmail.com/ does not fix this issue (#6850) since it only deals with hls_read_seek.