Opened 4 years ago

Closed 3 years ago

#9010 closed defect (fixed)

ffmpeg built with schannel freezes when outputting HLS with http_persistent

Reported by: r1ch Owned by:
Priority: important Component: avformat
Version: git-master Keywords: hls schannel regression
Cc: RytoEX@gmail.com, dillon@vodbox.io, matt@wizardcm.com, ushadow, tfoucu@gmail.com Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

When outputting HLS to a HTTPS URL using http_persistent, if ffmpeg is built with the schannel TLS backend (Windows), it will freeze as it writes the first segment. Switching the TLS backend to gnutls allows this to work as expected.

How to reproduce (Windows only):

Build ffmpeg with --enable-schannel. Stream some input file to Youtube with the HLS ingest endpoint (or any HLS server of your choice). You must use -http_persistent 1 to reproduce the issue.

% ffmpeg -i "bbb_sunflower_1080p_60fps_normal.mp4" -acodec aac -vcodec libx264 -vb 2.5M -f hls -hls_time 4 -http_persistent 1 "https://a.upload.youtube.com/http_upload_hls?cid=VALID_YOUTUBE_STREAM_KEY&copy=0&file=01.ts"

ffmpeg version N-100057-g76a99b2da0-ffmpeg-windows-build-helpers Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 10.1.0 (GCC)
  configuration: --pkg-config=pkg-config --pkg-config-flags=--static --extra-version=ffmpeg-windows-build-helpers --enable-version3 --disable-debug --disable-w32threads --arch=x86_64 --target-os=mingw32 --cross-prefix=/home/r1ch/ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32- --enable-libcaca --enable-gray --enable-libtesseract --enable-fontconfig --enable-gmp --enable-schannel --enable-libass --enable-libbluray --enable-libbs2b --enable-libflite --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libvorbis --enable-libwebp --enable-libzimg --enable-libzvbi --enable-libmysofa --enable-libopenjpeg --enable-libopenh264 --enable-liblensfun --enable-libvmaf --enable-libsrt --enable-libaribb24 --enable-demuxer=dash --enable-libxml2 --enable-opengl --enable-libdav1d --enable-cuda-llvm --enable-libaom --enable-libvpx --enable-nvenc --enable-nvdec --extra-libs=-lharfbuzz --extra-libs=-lm --extra-libs=-lpthread --extra-cflags=-DLIBTWOLAME_STATIC --extra-cflags=-DMODPLUG_STATIC --extra-cflags=-DCACA_STATIC --enable-amf --enable-libmfx --enable-gpl --enable-frei0r --enable-filter=frei0r --enable-librubberband --enable-libvidstab --enable-libx264 --enable-libx265 --enable-libxvid --enable-libdavs2 --enable-libxavs2 --enable-libxavs --enable-avresample --extra-cflags='-mtune=generic' --extra-cflags=-O3 --enable-shared --disable-static --prefix=/home/r1ch/ffmpeg-windows-build-helpers/sandbox/win64/ffmpeg_git_shared
  libavutil      56. 61.100 / 56. 61.100
  libavcodec     58.113.100 / 58.113.100
  libavformat    58. 64.100 / 58. 64.100
  libavdevice    58. 11.103 / 58. 11.103
  libavfilter     7. 90.100 /  7. 90.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb_sunflower_1080p_60fps_normal.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 1
    compatible_brands: isomavc1
    creation_time   : 2013-12-16T17:59:32.000000Z
    title           : Big Buck Bunny, Sunflower version
    artist          : Blender Foundation 2008, Janus Bager Kristensen 2013
    comment         : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
    genre           : Animation
    composer        : Sacha Goedegebure
  Duration: 00:10:34.53, start: 0.000000, bitrate: 4486 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 4001 kb/s, 60 fps, 60 tbr, 60k tbn, 120 tbc (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:32.000000Z
      handler_name    : GPAC ISO Video Handler
    Stream #0:1(und): Audio: mp3 (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 160 kb/s (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:37.000000Z
      handler_name    : GPAC ISO Audio Handler
    Stream #0:2(und): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side), fltp, 320 kb/s (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:37.000000Z
      handler_name    : GPAC ISO Audio Handler
    Side data:
      audio service type: main
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:2 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 000001c1ed118680] using SAR=1/1
[libx264 @ 000001c1ed118680] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 000001c1ed118680] profile High, level 4.2, 4:2:0, 8-bit
[libx264 @ 000001c1ed118680] 264 - core 161 r3020M d198931 - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=34 lookahead_threads=5 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=2500 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[aac @ 000001c1edec98c0] Using a PCE to encode channel layout "5.1(side)"
[hls muxer @ 000001c1ed116000] No HTTP method set, hls muxer defaulting to method PUT.
Output #0, hls, to 'https://a.upload.youtube.com/http_upload_hls?cid=xxxx&copy=0&file=01.ts':
  Metadata:
    major_brand     : isom
    minor_version   : 1
    compatible_brands: isomavc1
    composer        : Sacha Goedegebure
    title           : Big Buck Bunny, Sunflower version
    artist          : Blender Foundation 2008, Janus Bager Kristensen 2013
    comment         : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
    genre           : Animation
    encoder         : Lavf58.64.100
    Stream #0:0(und): Video: h264 (libx264), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 2500 kb/s, 60 fps, 90k tbn, 60 tbc (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:32.000000Z
      handler_name    : GPAC ISO Video Handler
      encoder         : Lavc58.113.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/2500000 buffer size: 0 vbv_delay: N/A
    Stream #0:1(und): Audio: aac (LC), 48000 Hz, 5.1(side), fltp, 394 kb/s (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:37.000000Z
      handler_name    : GPAC ISO Audio Handler
      encoder         : Lavc58.113.100 aac
    Side data:
      audio service type: main
[hls @ 000001c1ed985800] Opening 'https://a.upload.youtube.com/http_upload_hls?cid=xxxx&copy=0&file=010.ts' for writing

(ffmpeg becomes frozen at this point)

Change History (17)

comment:1 by Jim, 4 years ago

I've bisected the bug to commit f267a2ac9c740c77cee2f2c5fd2bd9bc048768b1 in the FFmpeg repository, related to #7975.

Last edited 4 years ago by Carl Eugen Hoyos (previous) (diff)

comment:2 by RytoEX, 4 years ago

Cc: RytoEX@gmail.com added

comment:3 by Dillon Pentz, 4 years ago

Cc: dillon@vodbox.io added

comment:4 by Matt, 4 years ago

Cc: matt@wizardcm.com added

comment:5 by Carl Eugen Hoyos, 4 years ago

Keywords: regression added; http_persistent removed
Priority: normal → important

comment:6 by ushadow, 3 years ago

I can reproduce the same problem with a simple nginx HTTP server with similar setup as in #7975. HTTP works. HTTPS with schannel and no http_persistent on Windows works. But HTTPS with schannel and http_persistent on Windows doesn't work. Compiled ffmpeg from latest snapshopt. Here is the log:

Log level: 48
Command line:
"C:\\cygwin64\\usr\\local\\bin\\ffmpeg.exe" -report -re -i bbb_sunflower_1080p_30fps_normal.ts -acodec copy -vcodec copy -g 60 -hls_flags +append_list -hls_list_size 0 -hls_playlist_type event -hls_time 6 -keyint_min 60 -method PUT -http_persistent 1 -sc_threshold 0 https://127.0.0.1/stream.m3u8
ffmpeg version N-103483-gf3f9041302 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (GCC)
  configuration: --arch=x86_64 --target-os=mingw32 --cross-prefix=x86_64-w64-mingw32- --prefix=/usr/local --pkg-config=pkg-config --pkg-config-flags=--static --extra-cflags=-static --extra-ldflags=-static --extra-libs='-lm -lz -fopenmp' --enable-static --disable-shared --enable-nonfree --enable-gpl --enable-libfdk-aac --enable-libfribidi --enable-libx264 --enable-schannel
  libavutil      57.  4.101 / 57.  4.101
  libavcodec     59.  7.100 / 59.  7.100
  libavformat    59.  5.100 / 59.  5.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8.  7.100 /  8.  7.100
  libswscale      6.  0.100 /  6.  0.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Splitting the commandline.
Reading option '-report' ... matched as option 'report' (generate a report) with argument '1'.
Reading option '-re' ... matched as option 're' (read input at native frame rate; equivalent to -readrate 1) with argument '1'.
Reading option '-i' ... matched as input url with argument 'bbb_sunflower_1080p_30fps_normal.ts'.
Reading option '-acodec' ... matched as option 'acodec' (force audio codec ('copy' to copy stream)) with argument 'copy'.
Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'copy'.
Reading option '-g' ... matched as AVOption 'g' with argument '60'.
Reading option '-hls_flags' ... matched as AVOption 'hls_flags' with argument '+append_list'.
Reading option '-hls_list_size' ... matched as AVOption 'hls_list_size' with argument '0'.
Reading option '-hls_playlist_type' ... matched as AVOption 'hls_playlist_type' with argument 'event'.
Reading option '-hls_time' ... matched as AVOption 'hls_time' with argument '6'.
Reading option '-keyint_min' ... matched as AVOption 'keyint_min' with argument '60'.
Reading option '-method' ... matched as AVOption 'method' with argument 'PUT'.
Reading option '-http_persistent' ... matched as AVOption 'http_persistent' with argument '1'.
Reading option '-sc_threshold' ... matched as AVOption 'sc_threshold' with argument '0'.
Reading option 'https://127.0.0.1/stream.m3u8' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option report (generate a report) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url bbb_sunflower_1080p_30fps_normal.ts.
Applying option re (read input at native frame rate; equivalent to -readrate 1) with argument 1.
Successfully parsed a group of options.
Opening an input file: bbb_sunflower_1080p_30fps_normal.ts.
[NULL @ 0000028ae8e09300] Opening 'bbb_sunflower_1080p_30fps_normal.ts' for reading
[file @ 0000028ae8e0a440] Setting default whitelist 'file,crypto,data'
[mpegts @ 0000028ae8e09300] Format mpegts probed with size=2048 and score=50
[mpegts @ 0000028ae8e09300] stream=0 stream_type=1b pid=100 prog_reg_desc=
[mpegts @ 0000028ae8e09300] stream=1 stream_type=f pid=101 prog_reg_desc=
[mpegts @ 0000028ae8e09300] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:2
[mpegts @ 0000028ae8e09300] probing stream 1 pp:2500
[mpegts @ 0000028ae8e09300] Probe with size=2758, packets=1 detected aac with score=51
[mpegts @ 0000028ae8e09300] probed stream 1
[extract_extradata @ 0000028aea8e5900] nal_unit_type: 9(AUD), nal_ref_idc: 0
[extract_extradata @ 0000028aea8e5900] nal_unit_type: 7(SPS), nal_ref_idc: 3
[extract_extradata @ 0000028aea8e5900] nal_unit_type: 8(PPS), nal_ref_idc: 3
[extract_extradata @ 0000028aea8e5900] nal_unit_type: 6(SEI), nal_ref_idc: 0
[extract_extradata @ 0000028aea8e5900] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0000028ae8e0c040] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 0000028ae8e0c040] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0000028ae8e0c040] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0000028ae8e0c040] nal_unit_type: 6(SEI), nal_ref_idc: 0
[h264 @ 0000028ae8e0c040] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0000028ae8e0c040] Format yuv420p chosen by get_format().
[h264 @ 0000028ae8e0c040] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 0000028ae8e0c040] no picture
[mpegts @ 0000028ae8e09300] max_analyze_duration 5000000 reached at 5013333 microseconds st:1
[mpegts @ 0000028ae8e09300] Packet had TEI flag set; marking as corrupt
[mpegts @ 0000028ae8e09300] After avformat_find_stream_info() pos: 0 bytes read:938128 seeks:2 frames:389
Guessed Channel Layout for Input Stream #0.1 : 5.1
Input #0, mpegts, from 'bbb_sunflower_1080p_30fps_normal.ts':
  Duration: 00:10:34.62, start: 1.400000, bitrate: 3600 kb/s
  Program 1
    Metadata:
      service_name    : Big Buck Bunny, Sunflower version
      service_provider: FFmpeg
  Stream #0:0[0x100], 151, 1/90000: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn
  Stream #0:1[0x101](und), 238, 1/90000: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, 5.1, fltp, 394 kb/s
Successfully opened the file.
Parsing a group of options: output url https://127.0.0.1/stream.m3u8.
Applying option acodec (force audio codec ('copy' to copy stream)) with argument copy.
Applying option vcodec (force video codec ('copy' to copy stream)) with argument copy.
Successfully parsed a group of options.
Opening an output file: https://127.0.0.1/stream.m3u8.
Successfully opened the file.
[mpegts @ 0000028aeb127d00] service 1 using PCR in pid=256, pcr_period=0ms
[mpegts @ 0000028aeb127d00] muxrate VBR, sdt every 1073741822000 ms, pat/pmt every 1073741822000 ms
[https @ 0000028aeac90a80] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[tcp @ 0000028aeaa80c80] Original list of addresses:
[tcp @ 0000028aeaa80c80] Address 127.0.0.1 port 443
[tcp @ 0000028aeaa80c80] Interleaved list of addresses:
[tcp @ 0000028aeaa80c80] Address 127.0.0.1 port 443
[tcp @ 0000028aeaa80c80] Starting connection attempt to 127.0.0.1 port 443
[tcp @ 0000028aeaa80c80] Successfully connected to 127.0.0.1 port 443
[https @ 0000028aeac90a80] request: GET /stream.m3u8 HTTP/1.1
User-Agent: Lavf/59.5.100
Accept: */*
Range: bytes=0-
Connection: close
Host: 127.0.0.1
Icy-MetaData: 1


[hls muxer @ 0000028aeb137f80] Found playlist sequence number: 0
[AVIOContext @ 0000028aeac91e00] Statistics: 583 bytes read, 0 seeks
Output #0, hls, to 'https://127.0.0.1/stream.m3u8':
  Metadata:
    encoder         : Lavf59.5.100
  Stream #0:0, 0, 1/90000: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 30 tbr, 90k tbn
  Stream #0:1(und), 0, 1/90000: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, 5.1, fltp, 394 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:1 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[NULL @ 0000028ae8e0c040] nal_unit_type: 7(SPS), nal_ref_idc: 3
[NULL @ 0000028ae8e0c040] nal_unit_type: 8(PPS), nal_ref_idc: 3
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[hls @ 0000028aeae79040] Opening 'https://127.0.0.1/stream9.ts' for writingx
[https @ 0000028aea870cc0] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[tcp @ 0000028aeb16a940] Original list of addresses:
[tcp @ 0000028aeb16a940] Address 127.0.0.1 port 443
[tcp @ 0000028aeb16a940] Interleaved list of addresses:
[tcp @ 0000028aeb16a940] Address 127.0.0.1 port 443
[tcp @ 0000028aeb16a940] Starting connection attempt to 127.0.0.1 port 443
[tcp @ 0000028aeb16a940] Successfully connected to 127.0.0.1 port 443
[https @ 0000028aea870cc0] request: PUT /stream9.ts HTTP/1.1
Transfer-Encoding: chunked
User-Agent: Lavf/59.5.100
Accept: */*
Connection: keep-alive
Host: 127.0.0.1
Icy-MetaData: 1



Received windows signal 0

Received windows signal 0
[tls @ 0000028aeb16a880] Unable to read from socket
[hls @ 0000028aeae79040] upload segment failed, will retry with a new http session.
[AVIOContext @ 0000028aeac920c0] Statistics: 0 seeks, 157 writeouts
[tls @ 0000028aeb16a880] Failed to send close message
[hls @ 0000028aeae79040] Opening 'https://127.0.0.1/stream9.ts' for writing
[https @ 0000028aea870cc0] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[tcp @ 0000028aeb16a940] Original list of addresses:
[tcp @ 0000028aeb16a940] Address 127.0.0.1 port 443
[tcp @ 0000028aeb16a940] Interleaved list of addresses:
[tcp @ 0000028aeb16a940] Address 127.0.0.1 port 443
[tcp @ 0000028aeb16a940] Starting connection attempt to 127.0.0.1 port 443}}}
Last edited 3 years ago by ushadow (previous) (diff)

comment:7 by ushadow, 3 years ago

Cc: ushadow added

comment:8 by Thierry, 3 years ago

Cc: tfoucu@gmail.com added

in reply to:  6 ; comment:9 by Steven Liu, 3 years ago

Replying to ushadow:

I can reproduce the same problem with a simple nginx HTTP server with similar setup as in #7975. HTTP works. HTTPS with schannel and no http_persistent on Windows works. But HTTPS with schannel and http_persistent on Windows doesn't work. Compiled ffmpeg from latest snapshopt. Here is the log:

Log level: 48
Command line:
"C:\\cygwin64\\usr\\local\\bin\\ffmpeg.exe" -report -re -i bbb_sunflower_1080p_30fps_normal.ts -acodec copy -vcodec copy -g 60 -hls_flags +append_list -hls_list_size 0 -hls_playlist_type event -hls_time 6 -keyint_min 60 -method PUT -http_persistent 1 -sc_threshold 0 https://127.0.0.1/stream.m3u8
ffmpeg version N-103483-gf3f9041302 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (GCC)
  configuration: --arch=x86_64 --target-os=mingw32 --cross-prefix=x86_64-w64-mingw32- --prefix=/usr/local --pkg-config=pkg-config --pkg-config-flags=--static --extra-cflags=-static --extra-ldflags=-static --extra-libs='-lm -lz -fopenmp' --enable-static --disable-shared --enable-nonfree --enable-gpl --enable-libfdk-aac --enable-libfribidi --enable-libx264 --enable-schannel
  libavutil      57.  4.101 / 57.  4.101
  libavcodec     59.  7.100 / 59.  7.100
  libavformat    59.  5.100 / 59.  5.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8.  7.100 /  8.  7.100
  libswscale      6.  0.100 /  6.  0.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Splitting the commandline.
Reading option '-report' ... matched as option 'report' (generate a report) with argument '1'.
Reading option '-re' ... matched as option 're' (read input at native frame rate; equivalent to -readrate 1) with argument '1'.
Reading option '-i' ... matched as input url with argument 'bbb_sunflower_1080p_30fps_normal.ts'.
Reading option '-acodec' ... matched as option 'acodec' (force audio codec ('copy' to copy stream)) with argument 'copy'.
Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'copy'.
Reading option '-g' ... matched as AVOption 'g' with argument '60'.
Reading option '-hls_flags' ... matched as AVOption 'hls_flags' with argument '+append_list'.
Reading option '-hls_list_size' ... matched as AVOption 'hls_list_size' with argument '0'.
Reading option '-hls_playlist_type' ... matched as AVOption 'hls_playlist_type' with argument 'event'.
Reading option '-hls_time' ... matched as AVOption 'hls_time' with argument '6'.
Reading option '-keyint_min' ... matched as AVOption 'keyint_min' with argument '60'.
Reading option '-method' ... matched as AVOption 'method' with argument 'PUT'.
Reading option '-http_persistent' ... matched as AVOption 'http_persistent' with argument '1'.
Reading option '-sc_threshold' ... matched as AVOption 'sc_threshold' with argument '0'.
Reading option 'https://127.0.0.1/stream.m3u8' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option report (generate a report) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url bbb_sunflower_1080p_30fps_normal.ts.
Applying option re (read input at native frame rate; equivalent to -readrate 1) with argument 1.
Successfully parsed a group of options.
Opening an input file: bbb_sunflower_1080p_30fps_normal.ts.
[NULL @ 0000028ae8e09300] Opening 'bbb_sunflower_1080p_30fps_normal.ts' for reading
[file @ 0000028ae8e0a440] Setting default whitelist 'file,crypto,data'
[mpegts @ 0000028ae8e09300] Format mpegts probed with size=2048 and score=50
[mpegts @ 0000028ae8e09300] stream=0 stream_type=1b pid=100 prog_reg_desc=
[mpegts @ 0000028ae8e09300] stream=1 stream_type=f pid=101 prog_reg_desc=
[mpegts @ 0000028ae8e09300] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:2
[mpegts @ 0000028ae8e09300] probing stream 1 pp:2500
[mpegts @ 0000028ae8e09300] Probe with size=2758, packets=1 detected aac with score=51
[mpegts @ 0000028ae8e09300] probed stream 1
[extract_extradata @ 0000028aea8e5900] nal_unit_type: 9(AUD), nal_ref_idc: 0
[extract_extradata @ 0000028aea8e5900] nal_unit_type: 7(SPS), nal_ref_idc: 3
[extract_extradata @ 0000028aea8e5900] nal_unit_type: 8(PPS), nal_ref_idc: 3
[extract_extradata @ 0000028aea8e5900] nal_unit_type: 6(SEI), nal_ref_idc: 0
[extract_extradata @ 0000028aea8e5900] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0000028ae8e0c040] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 0000028ae8e0c040] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0000028ae8e0c040] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0000028ae8e0c040] nal_unit_type: 6(SEI), nal_ref_idc: 0
[h264 @ 0000028ae8e0c040] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0000028ae8e0c040] Format yuv420p chosen by get_format().
[h264 @ 0000028ae8e0c040] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 0000028ae8e0c040] no picture
[mpegts @ 0000028ae8e09300] max_analyze_duration 5000000 reached at 5013333 microseconds st:1
[mpegts @ 0000028ae8e09300] Packet had TEI flag set; marking as corrupt
[mpegts @ 0000028ae8e09300] After avformat_find_stream_info() pos: 0 bytes read:938128 seeks:2 frames:389
Guessed Channel Layout for Input Stream #0.1 : 5.1
Input #0, mpegts, from 'bbb_sunflower_1080p_30fps_normal.ts':
  Duration: 00:10:34.62, start: 1.400000, bitrate: 3600 kb/s
  Program 1
    Metadata:
      service_name    : Big Buck Bunny, Sunflower version
      service_provider: FFmpeg
  Stream #0:0[0x100], 151, 1/90000: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn
  Stream #0:1[0x101](und), 238, 1/90000: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, 5.1, fltp, 394 kb/s
Successfully opened the file.
Parsing a group of options: output url https://127.0.0.1/stream.m3u8.
Applying option acodec (force audio codec ('copy' to copy stream)) with argument copy.
Applying option vcodec (force video codec ('copy' to copy stream)) with argument copy.
Successfully parsed a group of options.
Opening an output file: https://127.0.0.1/stream.m3u8.
Successfully opened the file.
[mpegts @ 0000028aeb127d00] service 1 using PCR in pid=256, pcr_period=0ms
[mpegts @ 0000028aeb127d00] muxrate VBR, sdt every 1073741822000 ms, pat/pmt every 1073741822000 ms
[https @ 0000028aeac90a80] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[tcp @ 0000028aeaa80c80] Original list of addresses:
[tcp @ 0000028aeaa80c80] Address 127.0.0.1 port 443
[tcp @ 0000028aeaa80c80] Interleaved list of addresses:
[tcp @ 0000028aeaa80c80] Address 127.0.0.1 port 443
[tcp @ 0000028aeaa80c80] Starting connection attempt to 127.0.0.1 port 443
[tcp @ 0000028aeaa80c80] Successfully connected to 127.0.0.1 port 443
[https @ 0000028aeac90a80] request: GET /stream.m3u8 HTTP/1.1
User-Agent: Lavf/59.5.100
Accept: */*
Range: bytes=0-
Connection: close
Host: 127.0.0.1
Icy-MetaData: 1


[hls muxer @ 0000028aeb137f80] Found playlist sequence number: 0
[AVIOContext @ 0000028aeac91e00] Statistics: 583 bytes read, 0 seeks
Output #0, hls, to 'https://127.0.0.1/stream.m3u8':
  Metadata:
    encoder         : Lavf59.5.100
  Stream #0:0, 0, 1/90000: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 30 tbr, 90k tbn
  Stream #0:1(und), 0, 1/90000: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, 5.1, fltp, 394 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:1 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[NULL @ 0000028ae8e0c040] nal_unit_type: 7(SPS), nal_ref_idc: 3
[NULL @ 0000028ae8e0c040] nal_unit_type: 8(PPS), nal_ref_idc: 3
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[hls @ 0000028aeae79040] Opening 'https://127.0.0.1/stream9.ts' for writingx
[https @ 0000028aea870cc0] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[tcp @ 0000028aeb16a940] Original list of addresses:
[tcp @ 0000028aeb16a940] Address 127.0.0.1 port 443
[tcp @ 0000028aeb16a940] Interleaved list of addresses:
[tcp @ 0000028aeb16a940] Address 127.0.0.1 port 443
[tcp @ 0000028aeb16a940] Starting connection attempt to 127.0.0.1 port 443
[tcp @ 0000028aeb16a940] Successfully connected to 127.0.0.1 port 443
[https @ 0000028aea870cc0] request: PUT /stream9.ts HTTP/1.1
Transfer-Encoding: chunked
User-Agent: Lavf/59.5.100
Accept: */*
Connection: keep-alive
Host: 127.0.0.1
Icy-MetaData: 1



Received windows signal 0

Received windows signal 0
[tls @ 0000028aeb16a880] Unable to read from socket
[hls @ 0000028aeae79040] upload segment failed, will retry with a new http session.
[AVIOContext @ 0000028aeac920c0] Statistics: 0 seeks, 157 writeouts
[tls @ 0000028aeb16a880] Failed to send close message
[hls @ 0000028aeae79040] Opening 'https://127.0.0.1/stream9.ts' for writing
[https @ 0000028aea870cc0] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[tcp @ 0000028aeb16a940] Original list of addresses:
[tcp @ 0000028aeb16a940] Address 127.0.0.1 port 443
[tcp @ 0000028aeb16a940] Interleaved list of addresses:
[tcp @ 0000028aeb16a940] Address 127.0.0.1 port 443
[tcp @ 0000028aeb16a940] Starting connection attempt to 127.0.0.1 port 443}}}

Try this patch please:

​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210913021204.22138-1-lq@chinaffmpeg.org/

in reply to:  9 comment:10 by RytoEX, 3 years ago

Replying to Steven Liu:

Try this patch please:

​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210913021204.22138-1-lq@chinaffmpeg.org/

I compiled the current FFmpeg git master with this patch applied and with --enable-schannel. I ran the command in the description and successfully streamed the full test video to YouTube via HLS on Windows.

This seems promising. Hopefully others can test and produce similar results.

comment:11 by Balling, 3 years ago

Is a) Failed to send close message normal?

b) is keep-alive between two domains allowed to happen?? See: ffplay ​https://play.itunes.apple.com/WebObjects/MZPlay.woa/hls/subscription/playlist.m3u8?cc=US&svcId=tvs.vds.4105&a=1580273278&isExternal=true&brandId=tvs.sbd.4000&id=337246031&l=en-US&aec=UHD&xtrick=true&webbrowser=true

"Cannot reuse HTTP connection for different host: vod-ap2-aoc.tv.apple.com:-1 != vod-ak-aoc.tv.apple.com:-1

[hls @ 00000282b0b67600] keepalive request failed for '​https://vod-ak-aoc.tv.apple.com/itunes-assets/VideoPreview125/v4/b7/a5/a6/b7a5a620-9977-e5f6-f242-7cb3939de3d3/P337246031_A1580273278_en_audio_gr32-.m3u8' with error: 'Invalid argument' when parsing playlist"

Audio in practice is from another host.

in reply to:  11 comment:12 by Steven Liu, 3 years ago

Replying to Balling:

Is a) Failed to send close message normal?

b) is keep-alive between two domains allowed to happen?? See: ffplay ​https://play.itunes.apple.com/WebObjects/MZPlay.woa/hls/subscription/playlist.m3u8?cc=US&svcId=tvs.vds.4105&a=1580273278&isExternal=true&brandId=tvs.sbd.4000&id=337246031&l=en-US&aec=UHD&xtrick=true&webbrowser=true

"Cannot reuse HTTP connection for different host: vod-ap2-aoc.tv.apple.com:-1 != vod-ak-aoc.tv.apple.com:-1

[hls @ 00000282b0b67600] keepalive request failed for '​https://vod-ak-aoc.tv.apple.com/itunes-assets/VideoPreview125/v4/b7/a5/a6/b7a5a620-9977-e5f6-f242-7cb3939de3d3/P337246031_A1580273278_en_audio_gr32-.m3u8' with error: 'Invalid argument' when parsing playlist"

Audio in practice is from another host.

That because -http_persistent 1 is default, you can try to use -http_persistent 0

BTW:
Is this problem happened with this ticket? Or should it merge into this ticket?

Last edited 3 years ago by Steven Liu (previous) (diff)

comment:13 by r1ch, 3 years ago

The patch in comment #comment:9 has solved the issue on our end.

Comment #comment:11 seems to be discussing issues with HLS input, I don't believe this is relevant to this ticket as the issue and patch only relate to HLS output.

Last edited 3 years ago by r1ch (previous) (diff)

comment:14 by ushadow, 3 years ago

Looks like the patch works. Is the patch going to be merged to Ffmpeg?

comment:15 by Balling, 3 years ago

Status: new → open

Does not help

ffplay.exe -http_persistent 0 ​https://events-delivery.apple.com/1402uekefjejgldkvaqrqxgjmtehwhez/vod_main_FmkzTaRUQHbdgFMjhhirKrarNznPoHEC/audio_ad_en_2ch_aac_128/prog_index.m3u8

Maybe you should build with openssl 3.0 that is finally compatible with gpl.

Yes, the patch should be merged if you will again confirm it works.

Last edited 3 years ago by Balling (previous) (diff)

in reply to:  15 comment:16 by RytoEX, 3 years ago

Replying to Balling:

Does not help

ffplay.exe -http_persistent 0 ​https://events-delivery.apple.com/1402uekefjejgldkvaqrqxgjmtehwhez/vod_main_FmkzTaRUQHbdgFMjhhirKrarNznPoHEC/audio_ad_en_2ch_aac_128/prog_index.m3u8

Maybe you should build with openssl 3.0 that is finally compatible with gpl.

Yes, the patch should be merged if you will again confirm it works.

The patch provided above is for fixing HLS output. The test case in this ticket is about outputting from FFmpeg to YouTube. You seem to be talking about HLS input by ingesting a playlist from some external source into FFmpeg. Your described scenario seems to be unrelated to the described scenario in this ticket or to the patch provided above.

Unfortunately, I'm not sure when I'll have time to rebuild FFmpeg and retest this. The ticket author and I did confirm earlier in comment:10 and comment:13 that the patch resolved the issue we're experiencing, however.

comment:17 by Steven Liu, 3 years ago

Resolution: → fixed
Status: open → closed

fixed in commit: f1c19867d72a14699277175101b2bcf1e333af88

Thanks for your test r1ch, RytoEX and ushadow

Note: See TracTickets for help on using tickets.