Potential "-rw_timeout" preset regression causing certain network streams failure

[Mad Sweeney]

ffmpeg stops with exit code 0 while reading certain hls http input streams.
It will stream for a varying amount of time and then stop.

To reproduce:

ffmpeg -report -v 9 -loglevel 99 -nostdin \
  -y -i 'http://as-hls-ww-live.akamaized.net/pool_904/live/ww/bbc_radio_three/bbc_radio_three.isml/bbc_radio_three-audio%3d96000.norewind.m3u8' \
  -to 3:00:00 \
  -acodec libmp3lame -write_xing 0 \
  testr3.mp3 \
  > testr3.out 2>&1

Also affects ffmpeg 7.0 and 7.1
Log is attached.

[MasterQuestionable]

͏    Probably similar?
͏    ​https://trac.ffmpeg.org/ticket/11108#comment:3

͏    I guess the remote end just terminated the connection normally.

[Mad Sweeney]

No, this is different from #11108.
ffmpeg terminates normally but continues streaming again when you immediately re-execute the same command.
Maybe ffmpeg could be made more robust by retrying the chunk or ignoring it.
Worked fine on Debian Bullseye (ffmpeg 4.3) but not on Bookworm (ffmpeg 5.1) or latest git.

[MasterQuestionable]

͏    Could you ensure it's reliably working in FFmpeg 4.3 but not later?
͏    Seems like server-side restriction whatsoever.

[Mad Sweeney]

I've confirmed that it workes and runs correctly to completion using 4.3.

Command line:
/usr/local/ffmpeg/432/ffmpeg -report -v 9 -loglevel 99 -nostdin -y -i "http://as-hls-ww-live.akamaized.net/pool_904/live/ww/bbc_radio_three/bbc_radio_three.isml/bbc_radio_three-audio%3d96000.norewind.m3u8" -to 3:00:00 -acodec libmp3lame -write_xing 0 testr3.mp3
ffmpeg version 4.3.2-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100

[MasterQuestionable]

͏    Try using User-Agent of FFmpeg 4.3 with later FFmpeg via "-headers". (or "-user_agent"?)
͏    ​https://trac.ffmpeg.org/ticket/11058#comment:2

͏    For debug interests, also compare the request headers.

[Mad Sweeney]

Hi,

Various experiments with headers did not resolve the problem.
However, I tried using a very large timeout with option:

-rw_timeout 2147483647

That seemed to cause the stream to run for much longer but still terminating early.
Then I tried the

-re

option and am having some success with that so far.
It could be that the later versions of ffmpeg are using a very small timeout resulting in stream termination.
A timeout of 0 seems to mean return immediately if no input available.
A timeout of -1 (to indicate no timeout) is rejected.
Maybe -1 should be accepted to mean not to timeout?
There is another open ticket which describes the same problem I'm having: #10718

[MasterQuestionable]

͏    Server-side restriction mostly likely:
͏    Excessive download disallowed.
https://ffmpeg.org/ffmpeg.html#Advanced-options
͏    “"-re" (input)
͏    Read input at native frame rate. This is equivalent to setting "-readrate 1".”

͏    Probably you shall use dedicated downloader (e.g. ​`yt-dlp`) for downloading?


͏    Per:
https://ffmpeg.org/ffmpeg-protocols.html#Protocols
͏    “"-rw_timeout"
͏    Maximum time to wait for (network) read/write operations to complete, in microseconds.”

͏    "-rw_timeout 2147483647" would mean 2,147,483.647 s (24::20:31:23.647).
͏    Likely not simply timeout but connection reset.

[Mad Sweeney]

Seems to be fixed.
Can no longer reproduce the problem with v7.1.3