Opened 2 months ago

Last modified 2 months ago

#8998 new defect

Getting "Error when loading first segment" in some of the version FFmpeg

Reported by: NabiKAZ Owned by:
Priority: important Component: avformat
Version: git-master Keywords: hls regression
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

I have remote index.m3u8 file and I want to download it with ffmpeg.
The contents file is:

#EXTM3U
#EXT-X-TARGETDURATION:6
#EXT-X-ALLOW-CACHE:YES
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-KEY:METHOD=AES-128,URI="https://example.com/encryption.key?x=AVCoPj..."
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:6.000,
https://example.com/seg-1-v1-a1.ts?x=AVCoPj...
#EXTINF:6.000,
https://example.com/seg-2-v1-a1.ts?x=AVCoPj...
#EXTINF:6.000,
https://example.com/seg-3-v1-a1.ts?x=AVCoPj...
#EXTINF:6.000,
:
:

I tried this command:

ffmpeg -headers "User-Agent: a" -i "https://example.com/index.m3u8?x=AVCoPj..." -c copy "video.mp4"

In the some of versions of ffmpeg I getting this errors:

[AVIOContext @ 000001cef2816ec0] Statistics: 368 bytes read, 0 seeks
[hls @ 000001cef21be3c0] Error when loading first segment 'https://example.com/seg-1-v1-a1.ts?x=AVCoPj...'
[AVIOContext @ 000001cef21c8f00] Statistics: 24167 bytes read, 0 seeks
https://example.com/index.m3u8?x=AVCoPj...: Invalid data found when processing input

You can see all logs here:

# ffmpeg -headers "User-Agent: a" -i "https://example.com/index.m3u8?x=AVCoPj..." -c copy "video.mp4" -loglevel debug
ffmpeg version 2020-11-18-git-e3081d6f4f-full_build-www.gyan.dev Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 10.2.0 (Rev3, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      56. 60.100 / 56. 60.100
  libavcodec     58.112.103 / 58.112.103
  libavformat    58. 64.100 / 58. 64.100
  libavdevice    58. 11.102 / 58. 11.102
  libavfilter     7. 90.100 /  7. 90.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
Splitting the commandline.
Reading option '-headers' ... matched as AVOption 'headers' with argument 'User-Agent: a'.
Reading option '-i' ... matched as input url with argument 'https://example.com/index.m3u8?x=AVCoPj...'.
Reading option '-c' ... matched as option 'c' (codec name) with argument 'copy'.
Reading option 'video.mp4' ... matched as output url.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url https://example.com/index.m3u8?x=AVCoPj....
Successfully parsed a group of options.
Opening an input file: https://example.com/index.m3u8?x=AVCoPj....
[NULL @ 000001cef21be3c0] Opening 'https://example.com/index.m3u8?x=AVCoPj...' for reading
[https @ 000001cef21bed80] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[https @ 000001cef21bed80] No trailing CRLF found in HTTP header. Adding it.
[tcp @ 000001cef21c1f80] Original list of addresses:
[tcp @ 000001cef21c1f80] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef21c1f80] Interleaved list of addresses:
[tcp @ 000001cef21c1f80] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef21c1f80] Starting connection attempt to 1.2.3.4 port 443
[tcp @ 000001cef21c1f80] Successfully connected to 1.2.3.4 port 443
[https @ 000001cef21bed80] request: GET /index.m3u8?x=AVCoPj... HTTP/1.1
Accept: */*
Range: bytes=0-
Connection: close
Host: example.com
Icy-MetaData: 1
User-Agent: a


[hls @ 000001cef21be3c0] Format hls probed with size=2048 and score=100
[hls @ 000001cef21be3c0] Skip ('#EXT-X-ALLOW-CACHE:YES')
[hls @ 000001cef21be3c0] Skip ('#EXT-X-VERSION:3')
[https @ 000001cef21bed80] Last chunk received, closing conn
[hls @ 000001cef21be3c0] HLS request for url 'https://example.com/seg-1-v1-a1.ts?x=AVCoPj...', offset 0, playlist 0
[hls @ 000001cef21be3c0] Opening 'https://example.com/encryption.key?x=AVCoPj...' for reading
[tcp @ 000001cef2891bc0] Original list of addresses:
[tcp @ 000001cef2891bc0] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef2891bc0] Interleaved list of addresses:
[tcp @ 000001cef2891bc0] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef2891bc0] Starting connection attempt to 1.2.3.4 port 443
[tcp @ 000001cef2891bc0] Successfully connected to 1.2.3.4 port 443
[https @ 000001cef2270900] request: GET /encryption.key?x=AVCoPj... HTTP/1.1
Accept: */*
Range: bytes=0-
Connection: keep-alive
Host: example.com
Icy-MetaData: 1
User-Agent: a


[AVIOContext @ 000001cef2269980] Statistics: 16 bytes read, 0 seeks
[hls @ 000001cef21be3c0] Opening 'crypto+https://example.com/seg-1-v1-a1.ts?x=AVCoPj...' for reading
[tcp @ 000001cef21eb740] Original list of addresses:
[tcp @ 000001cef21eb740] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef21eb740] Interleaved list of addresses:
[tcp @ 000001cef21eb740] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef21eb740] Starting connection attempt to 1.2.3.4 port 443
[tcp @ 000001cef21eb740] Successfully connected to 1.2.3.4 port 443
[https @ 000001cef282acc0] request: GET /seg-1-v1-a1.ts?x=AVCoPj... HTTP/1.1
Accept: */*
Range: bytes=0-
Connection: keep-alive
Host: example.com
Icy-MetaData: 1
User-Agent: a


[AVIOContext @ 000001cef28992c0] Statistics: 368 bytes read, 0 seeks
[hls @ 000001cef21be3c0] HLS request for url 'https://example.com/seg-2-v1-a1.ts?x=AVCoPj...', offset 0, playlist 0
[hls @ 000001cef21be3c0] Opening 'crypto+https://example.com/seg-2-v1-a1.ts?x=AVCoPj...' for reading
[tcp @ 000001cef21e2a80] Original list of addresses:
[tcp @ 000001cef21e2a80] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef21e2a80] Interleaved list of addresses:
[tcp @ 000001cef21e2a80] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef21e2a80] Starting connection attempt to 1.2.3.4 port 443
[tcp @ 000001cef21e2a80] Successfully connected to 1.2.3.4 port 443
[https @ 000001cef282acc0] request: GET /seg-2-v1-a1.ts?x=AVCoPj... HTTP/1.1
Accept: */*
Range: bytes=0-
Connection: keep-alive
Host: example.com
Icy-MetaData: 1
User-Agent: a


[AVIOContext @ 000001cef21c6d40] Statistics: 368 bytes read, 0 seeks
[hls @ 000001cef21be3c0] HLS request for url 'https://example.com/seg-3-v1-a1.ts?x=AVCoPj...', offset 0, playlist 0
[hls @ 000001cef21be3c0] Opening 'crypto+https://example.com/seg-3-v1-a1.ts?x=AVCoPj...' for reading
[tcp @ 000001cef282bd40] Original list of addresses:
[tcp @ 000001cef282bd40] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef282bd40] Interleaved list of addresses:
[tcp @ 000001cef282bd40] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef282bd40] Starting connection attempt to 1.2.3.4 port 443
[tcp @ 000001cef282bd40] Successfully connected to 1.2.3.4 port 443
[https @ 000001cef2270900] request: GET /seg-3-v1-a1.ts?x=AVCoPj... HTTP/1.1
Accept: */*
Range: bytes=0-
Connection: keep-alive
Host: example.com
Icy-MetaData: 1
User-Agent: a


[AVIOContext @ 000001cef226e6c0] Statistics: 368 bytes read, 0 seeks
[hls @ 000001cef21be3c0] HLS request for url 'https://example.com/seg-4-v1-a1.ts?x=AVCoPj...', offset 0, playlist 0
[hls @ 000001cef21be3c0] Opening 'crypto+https://example.com/seg-4-v1-a1.ts?x=AVCoPj...' for reading
[tcp @ 000001cef282bec0] Original list of addresses:
[tcp @ 000001cef282bec0] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef282bec0] Interleaved list of addresses:
[tcp @ 000001cef282bec0] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef282bec0] Starting connection attempt to 1.2.3.4 port 443
[tcp @ 000001cef282bec0] Successfully connected to 1.2.3.4 port 443
[https @ 000001cef282acc0] request: GET /seg-4-v1-a1.ts?x=AVCoPj... HTTP/1.1
Accept: */*
Range: bytes=0-
Connection: keep-alive
Host: example.com
Icy-MetaData: 1
User-Agent: a

:
:
:

[AVIOContext @ 000001cef226b9c0] Statistics: 368 bytes read, 0 seeks
[hls @ 000001cef21be3c0] HLS request for url 'https://example.com/seg-75-v1-a1.ts?x=AVCoPj...', offset 0, playlist 0
[hls @ 000001cef21be3c0] Opening 'crypto+https://example.com/seg-75-v1-a1.ts?x=AVCoPj...' for reading
[tcp @ 000001cef28dd1c0] Original list of addresses:
[tcp @ 000001cef28dd1c0] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef28dd1c0] Interleaved list of addresses:
[tcp @ 000001cef28dd1c0] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef28dd1c0] Starting connection attempt to 1.2.3.4 port 443
[tcp @ 000001cef28dd1c0] Successfully connected to 1.2.3.4 port 443
[https @ 000001cef282ba40] request: GET /seg-75-v1-a1.ts?x=AVCoPj... HTTP/1.1
Accept: */*
Range: bytes=0-
Connection: keep-alive
Host: example.com
Icy-MetaData: 1
User-Agent: a


[AVIOContext @ 000001cef2817800] Statistics: 368 bytes read, 0 seeks
[hls @ 000001cef21be3c0] HLS request for url 'https://example.com/seg-76-v1-a1.ts?x=AVCoPj...', offset 0, playlist 0
[hls @ 000001cef21be3c0] Opening 'crypto+https://example.com/seg-76-v1-a1.ts?x=AVCoPj...' for reading
[tcp @ 000001cef2270700] Original list of addresses:
[tcp @ 000001cef2270700] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef2270700] Interleaved list of addresses:
[tcp @ 000001cef2270700] Address 1.2.3.4 port 443
    Last message repeated 1 times
[tcp @ 000001cef2270700] Starting connection attempt to 1.2.3.4 port 443
[tcp @ 000001cef2270700] Successfully connected to 1.2.3.4 port 443
[https @ 000001cef2271600] request: GET /seg-76-v1-a1.ts?x=AVCoPj... HTTP/1.1
Accept: */*
Range: bytes=0-
Connection: keep-alive
Host: example.com
Icy-MetaData: 1
User-Agent: a


[AVIOContext @ 000001cef2816ec0] Statistics: 368 bytes read, 0 seeks
[hls @ 000001cef21be3c0] Error when loading first segment 'https://example.com/seg-1-v1-a1.ts?x=AVCoPj...'
[AVIOContext @ 000001cef21c8f00] Statistics: 24167 bytes read, 0 seeks
https://example.com/index.m3u8?x=AVCoPj...: Invalid data found when processing input

I tried many versions of ffmpeg and get these results:

ffmpeg-1.0.1-win64-static		Error in the pull function.
ffmpeg-1.2-win64-static			Error in the pull function.
ffmpeg-2.0.1-win64-static		Error in the pull function.
ffmpeg-2.2.3-win64-static		Error in the pull function.
ffmpeg-3.0-win64-static			OK
ffmpeg-3.1.5-win64-static		OK warning...
ffmpeg-3.2.4-win64-static		OK
ffmpeg-3.3.4-win64-static		OK
ffmpeg-3.4.2-win64-static		OK
ffmpeg-4.0.2-win64-static		OK
ffmpeg-4.1.4-win64-static		OK
ffmpeg-4.2.3-win64-static		OK
ffmpeg-4.3-win64-static			Error when loading first segment
ffmpeg-4.3.1-win64-static		Error when loading first segment
ffmpeg-2020-11-18-git-e3081d6f4f	Error when loading first segment

Seem after >=4.3 there is a bug in ffmpeg!

Change History (5)

comment:2 Changed 2 months ago by cehoyos

  • Component changed from ffmpeg to avformat
  • Keywords hls regression added; bug ffmpeg m3u8 removed
  • Priority changed from important to normal

If you cannot provide a sample url you will have to run git bisect yourself.

comment:3 Changed 2 months ago by NabiKAZ

The final result of git bisect command:

# git bisect good
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[5e829262a6a6068dfed9d76da4a9635f1f4be391] lavf/hls: add http_seekable option for HTTP partial requests

# git bisect bad
5e829262a6a6068dfed9d76da4a9635f1f4be391 is the first bad commit
commit 5e829262a6a6068dfed9d76da4a9635f1f4be391
Author: Jun Zhao <barryjzhao@tencent.com>
Date:   Thu Aug 8 00:12:16 2019 +0800

    lavf/hls: add http_seekable option for HTTP partial requests

    Add http_seekable option for HTTP partial requests, when The
    EXT-X-BYTERANGE tag indicates that a Media Segment is a sub-range
    of the resource identified by its URI, we can use HTTP partial
    requests to get the Media Segment.

    Reviewed-by: Steven Liu <lq@chinaffmpeg.org>
    Signed-off-by: Jun Zhao <barryjzhao@tencent.com>

 doc/demuxers.texi | 4 ++++
 libavformat/hls.c | 9 +++++++--
 2 files changed, 11 insertions(+), 2 deletions(-)

https://github.com/FFmpeg/FFmpeg/commit/5e829262a6a6068dfed9d76da4a9635f1f4be391

comment:4 Changed 2 months ago by cehoyos

  • Priority changed from normal to important
  • Version changed from unspecified to git-master

comment:5 Changed 2 months ago by NabiKAZ

Note: Also in the same system on the chrome browser nothing problem with play this video URL and also can be download with streamlink software.(so not have any problem for network or blocking IP or broken link.)

Last edited 2 months ago by NabiKAZ (previous) (diff)
Note: See TracTickets for help on using tickets.