Seeking to beginning of HLS stream fails

[tospi]

I created an HLS stream with the following command:

E:\ffmpeg.exe -lavfi testsrc2=duration=30 hls/file.m3u8

I played the stream with the following command:

E:\ffplay.exe hls/file.m3u8

When I try to seek to the beginning (left arrow key pressed once during the first seconds), then the playback jumps to the beginning of the second segment (00:00:10) instead of the beginning of the stream (00:00:00). In addition, the playback speed is somehow slower then normal after this seek.

My assumption is that the start offset is somehow wrong and this is causing the seek to fail. I am not sure if this is a problem during stream creation (muxer) or only on the playback side (demuxer?). I assume a problem on the playback side, because the seek is working in VLC 2.2.6.

I also tried opening the HLS stream via a webserver, but there is no difference:

E:\ffplay.exe http://192.168.0.101:8085/hls/file.m3u8

Output of stream creation:

% ffmpeg.exe -lavfi testsrc2=duration=30 hls/file.m3u8 -v verbose
ffmpeg version N-89127-g8f4702a93f Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      56.  0.100 / 56.  0.100
  libavcodec     58.  3.103 / 58.  3.103
  libavformat    58.  2.100 / 58.  2.100
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  2.100 /  7.  2.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
[Parsed_testsrc2_0 @ 00000267df8aa760] size:320x240 rate:25/1 duration:30.000000 sar:1/1
Stream mapping:
  testsrc2 -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
[Parsed_testsrc2_0 @ 00000267df8aaa40] size:320x240 rate:25/1 duration:30.000000 sar:1/1
[libx264 @ 00000267df8ad640] using SAR=1/1
[libx264 @ 00000267df8ad640] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 00000267df8ad640] profile High, level 1.3
[libx264 @ 00000267df8ad640] 264 - core 152 r2851 ba24899 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=7 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[hls @ 00000267df8ab380] Opening 'hls/file0.ts' for writing
[mpegts @ 00000267df974fe0] muxrate VBR, pcr every 2 pkts, sdt every 2147483647, pat/pmt every 2147483647 pkts
Output #0, hls, to 'hls/file.m3u8':
  Metadata:
    encoder         : Lavf58.2.100
    Stream #0:0: Video: h264 (libx264), 1 reference frame, yuv420p, 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 25 fps, 90k tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc58.3.103 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
[hls @ 00000267df8ab380] Opening 'hls/file1.ts' for writing
[hls @ 00000267df8ab380] Opening 'hls/file.m3u8.tmp' for writing
[hls muxer @ 00000267df8ab980] EXT-X-MEDIA-SEQUENCE:0
[hls @ 00000267df8ab380] Opening 'hls/file2.ts' for writing
[hls @ 00000267df8ab380] Opening 'hls/file.m3u8.tmp' for writing
[hls muxer @ 00000267df8ab980] EXT-X-MEDIA-SEQUENCE:0
[Parsed_testsrc2_0 @ 00000267df8aaa40] EOF timestamp not reliablespeed=41.7x
No more output streams to write to, finishing.
[hls @ 00000267df8ab380] Opening 'hls/file.m3u8.tmp' for writing
[hls muxer @ 00000267df8ab980] EXT-X-MEDIA-SEQUENCE:0
frame=  750 fps=0.0 q=-1.0 Lsize=N/A time=00:00:29.88 bitrate=N/A speed=43.2x
video:1019kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file #0 (hls/file.m3u8):
  Output stream #0:0 (video): 750 frames encoded; 750 packets muxed (1043850 bytes);
  Total: 750 packets (1043850 bytes) muxed
[libx264 @ 00000267df8ad640] frame I:3     Avg QP:18.79  size:  5454
[libx264 @ 00000267df8ad640] frame P:215   Avg QP:26.80  size:  2011
[libx264 @ 00000267df8ad640] frame B:532   Avg QP:31.45  size:  1117
[libx264 @ 00000267df8ad640] consecutive B-frames:  2.5%  5.9%  8.4% 83.2%
[libx264 @ 00000267df8ad640] mb I  I16..4: 35.6% 37.0% 27.4%
[libx264 @ 00000267df8ad640] mb P  I16..4:  2.4%  5.0%  1.2%  P16..4: 13.7% 10.2%  7.0%  0.0%  0.0%    skip:60.4%
[libx264 @ 00000267df8ad640] mb B  I16..4:  0.2%  0.3%  0.1%  B16..8: 19.9%  7.4%  2.0%  direct: 2.6%  skip:67.4%  L0:52.8% L1:41.7% BI: 5.5%
[libx264 @ 00000267df8ad640] 8x8 transform intra:54.3% inter:30.9%
[libx264 @ 00000267df8ad640] coded y,uvDC,uvAC intra: 10.1% 28.2% 23.9% inter: 5.4% 15.2% 12.6%
[libx264 @ 00000267df8ad640] i16 v,h,dc,p: 70% 25%  6%  0%
[libx264 @ 00000267df8ad640] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  6%  9% 84%  1%  0%  0%  0%  0%  0%
[libx264 @ 00000267df8ad640] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 40% 21% 34%  1%  1%  1%  1%  1%  1%
[libx264 @ 00000267df8ad640] i8c dc,h,v,p: 53% 17% 29%  1%
[libx264 @ 00000267df8ad640] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 00000267df8ad640] ref P L0: 58.3%  7.8% 20.0% 13.9%
[libx264 @ 00000267df8ad640] ref B L0: 79.7% 16.4%  3.9%
[libx264 @ 00000267df8ad640] ref B L1: 92.1%  7.9%
[libx264 @ 00000267df8ad640] kb/s:278.18

Output of playback:

% ffplay.exe hls/file.m3u8 -v verbose
ffplay version N-89127-g8f4702a93f Copyright (c) 2003-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      56.  0.100 / 56.  0.100
  libavcodec     58.  3.103 / 58.  3.103
  libavformat    58.  2.100 / 58.  2.100
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  2.100 /  7.  2.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Initialized direct3d renderer.
[hls,applehttp @ 000001f4de260060] HLS request for url 'hls/file0.ts', offset 0, playlist 0
[hls,applehttp @ 000001f4de260060] Opening 'hls/file0.ts' for reading
[h264 @ 000001f4de263be0] Reinit context to 320x240, pix_fmt: yuv420p
[hls,applehttp @ 000001f4de260060] max_analyze_duration 5000000 reached at 5000000 microseconds st:0
Input #0, hls,applehttp, from 'hls/file.m3u8':
  Duration: 00:00:30.00, start: 1.480000, bitrate: 0 kb/s
  Program 0
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Video: h264 (High), 1 reference frame ([27][0][0][0] / 0x001B), yuv420p(left), 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Metadata:
      variant_bitrate : 0
[h264 @ 000001f4e3627700] Reinit context to 320x240, pix_fmt: yuv420p
[ffplay_buffer @ 000001f4de277fe0] w:320 h:240 pixfmt:yuv420p tb:1/90000 fr:25/1 sar:1/1 sws_param:
Created 320x240 texture with SDL_PIXELFORMAT_IYUV.
[hls,applehttp @ 000001f4de260060] HLS request for url 'hls/file0.ts', offset 0, playlist 0
[hls,applehttp @ 000001f4de260060] Opening 'hls/file0.ts' for reading
[hls,applehttp @ 000001f4de260060] HLS request for url 'hls/file1.ts', offset 0, playlist 0
[hls,applehttp @ 000001f4de260060] Opening 'hls/file1.ts' for reading
[h264 @ 000001f4e3627700] Reinit context to 320x240, pix_fmt: yuv420p
[ffplay_buffer @ 000001f4de31ef20] w:320 h:240 pixfmt:yuv420p tb:1/90000 fr:25/1 sar:1/1 sws_param:
   4.15 M-V: -8.688 fd=   0 aq=    0KB vq=   34KB sq=    0B f=0/0
   5.90 M-V: -7.807 fd=   0 aq=    0KB vq=   35KB sq=    0B f=0/0

[Steven Liu]

ffmpeg -f lavfi -i testsrc2=duration=30 -vcodec mpeg2video -hls_time 10 hls/file.m3u8

use mpeg2video codec is ok.

[tospi]

Replying to stevenliu:

ffmpeg -f lavfi -i testsrc2=duration=30 -vcodec mpeg2video -hls_time 10 hls/file.m3u8

use mpeg2video codec is ok.

Mpeg2 is too old and has bad quality, so this is not really a solution...

With HEVC/H.265, the seeking problem is also the same by the way.

[Steven Liu]

Replying to tospi:

Replying to stevenliu:

ffmpeg -f lavfi -i testsrc2=duration=30 -vcodec mpeg2video -hls_time 10 hls/file.m3u8

use mpeg2video codec is ok.

Mpeg2 is too old and has bad quality, so this is not really a solution...

With HEVC/H.265, the seeking problem is also the same by the way.

Yes, i just want say, maybe need to process codec or mpegts part :D

[Steven Liu]

Mark process flag:
look at the phenomenon， looks like seek cannot seek to the first keyframe:
test command: ./ffmpeg -f lavfi -i testsrc2=duration=10 -r:v 25 -g 2 -hls_time 3 hls/file.m3u8
it seek to the second frame, 0.080s 40ms per frame. keyframe interval is 2

[Steven Liu]

Replying to tospi:

Replying to stevenliu:

ffmpeg -f lavfi -i testsrc2=duration=30 -vcodec mpeg2video -hls_time 10 hls/file.m3u8

use mpeg2video codec is ok.

Mpeg2 is too old and has bad quality, so this is not really a solution...

With HEVC/H.265, the seeking problem is also the same by the way.

try this patch please:
​https://patchwork.ffmpeg.org/patch/7076/

[Balling]

See ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210525125238.325548-1-gustav.grusell@gmail.com/ and ​​https://patchwork.ffmpeg.org/patch/7076/

Should fix this, second was NAK'ed but I suppose there may be some sense there too.

Also #7485 is a duplicate.

[Gustav Grusell]

As far as I can tell #7485 is not a duplicate even though it it very similar. Different types of seek is used in the two scenarios: In the scenario above, ffplay will use the seek_frame_byte funtion for seeking, while in the scenario of #7485 hls_read_seek is used.

Related to the above, the patch ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210525125238.325548-1-gustav.grusell@gmail.com/ does not fix this issue (#6850) since it only deals with hls_read_seek.

[Balling]

This is now fixed with -bytes 0 workaround after #7485 is fixed with e78d0810d1741535c95e5ae0f198977626b1cdff.

-bytes 0 is for ffplay, if you did not get it.

e78d0810d1741535c95e5ae0f198977626b1cdff also fixes some parsing of avc, as

[h264 @ 000001cb69bb5e40] mmco: unref short failure

shows, very nice!

[Balling]

ffmpeg -f lavfi -i testsrc2=duration=30 -vcodec mpeg2video -hls_time 10 hls/file.m3u8

use mpeg2video codec is ok.

False. Same behaviour.

Before e78d0810d1741535c95e5ae0f198977626b1cdff even with -bytes 0 you could not seek before 00:00:00.480!

[Balling]

I think this may happen due to my ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210604062818.2099-1-val.zapod.vz@gmail.com/
"circuiting force will never be checked" in case SEEK_END and here is the bug.

Anyway, there are still two bugs even with -bytes 0: first of all when you seek right from 0:00:00.000 you are supposed to be on 10 seconds, right? And yet you are now on 20 seconds! Does not happen on mpeg2video.

Also when you seek left on 25 seconds you ARE FIRST seeking to 10 seconds start of segment and only then you seek to 15 seconds, very crazy. And that can be verified on pause but even without it looks obvious. Last one happens on mpeg2video too.

[Balling]

Considering -bytes 0 is now default [for hls only] (and 1 is even disallowed for hls) after 92053aa26053b941a027a4fc56674d7d86ba1e58 it should be fixed, will test. I am not sure it is such a good idea to disallow byte seek, since that will cause speed penalties... But whatever.

[Balling]

Fixed in 92053aa26053b941a027a4fc56674d7d86ba1e58 (hevc, avc, mpeg2). Very nice! -bytes 1 does not allow for seek either now.

[Balling]

Please note that seeking is still broken in fmp4 though.

​http://vectronic.io/hls_seek_issue/ts/in.m3u8 works good, one can seek to the start

​http://vectronic.io/hls_seek_issue/fmp4/in.m3u8 bad, cannot seek at all. This is BTW now much worse with 92053aa26053b941a027a4fc56674d7d86ba1e58!! It cannot seek now at all, while seeking using bytes was at least doing something if only seeking forward instead of backward, LOL. Patch should be applied asap!