ffmpeg do unauthorized requests with auth_type basic

[sergey]

Summary of the bug:
In scope of HLS authentication testing it was found that ffmpeg (LibAV) uses automatic 2 step authorization:
LibAV can use parameter auth_type that might help with 2 step authorizations by setting HTTP authentication type:
none - Choose the HTTP authentication type automatically. This is the default.
basic - Choose the HTTP basic authentication.
But in reality in any case (for example with auth_type basic) LibAV sometimes do unauthorized requests. From nginx log it looks like:

192.168.0.12 - kraken [16/Feb/2021:14:50:22 +0300] "GET /hls/test.m3u8 HTTP/1.1" 206 415 "-" "Lavf/58.45.100"
192.168.0.12 - - [16/Feb/2021:14:50:22 +0300] "GET /hls/segment013.ts HTTP/1.1" 401 195 "-" "Lavf/58.45.100"
192.168.0.12 - - [16/Feb/2021:14:50:22 +0300] "GET /hls/segment014.ts HTTP/1.1" 401 195 "-" "Lavf/58.45.100"
192.168.0.12 - kraken [16/Feb/2021:14:50:22 +0300] "GET /hls/segment013.ts HTTP/1.1" 206 5707680 "-" "Lavf/58.45.100"
192.168.0.12 - kraken [16/Feb/2021:14:50:24 +0300] "GET /hls/segment014.ts HTTP/1.1" 206 6059052 "-" "Lavf/58.45.100"
192.168.0.12 - kraken [16/Feb/2021:14:50:25 +0300] "GET /hls/segment015.ts HTTP/1.1" 206 5573260 "-" "Lavf/58.45.100"
192.168.0.12 - kraken [16/Feb/2021:14:50:25 +0300] "GET /hls/segment016.ts HTTP/1.1" 206 6371696 "-" "Lavf/58.45.100"
192.168.0.12 - kraken [16/Feb/2021:14:50:25 +0300] "GET /hls/segment017.ts HTTP/1.1" 206 5536976 "-" "Lavf/58.45.100"
192.168.0.12 - kraken [16/Feb/2021:14:50:26 +0300] "GET /hls/segment019.ts HTTP/1.1" 206 5866728 "-" "Lavf/58.45.100"
192.168.0.12 - kraken [16/Feb/2021:14:50:26 +0300] "GET /hls/segment018.ts HTTP/1.1" 206 5918804 "-" "Lavf/58.45.100"
192.168.0.12 - kraken [16/Feb/2021:14:50:26 +0300] "GET /hls/segment020.ts HTTP/1.1" 206 6353648 "-" "Lavf/58.45.100"
192.168.0.12 - kraken [16/Feb/2021:14:50:27 +0300] "GET /hls/segment021.ts HTTP/1.1" 206 5902824 "-" "Lavf/58.45.100"
192.168.0.12 - kraken [16/Feb/2021:14:50:27 +0300] "GET /hls/segment022.ts HTTP/1.1" 206 3860392 "-" "Lavf/58.45.100"

It might be a problem somewhere in request context creating. Sometimes it uses auth_type paramater, but sometimes it doesn't:

libavformat\avio.c

int ffurl_open_whitelist(URLContext **puc, const char *filename, int flags,
                         const AVIOInterruptCB *int_cb, AVDictionary **options,
                         const char *whitelist, const char* blacklist,
                         URLContext *parent)
{
..........................
if (options && (*puc)->prot->priv_data_class &&
    (ret = av_opt_set_dict((*puc)->priv_data, options)) < 0)

And then it leads to missed authorization header in requests:

Replace auth type:
void ff_http_auth_handle_header(HTTPAuthState *state, const char *key,
                                const char *value)
..................
if (av_stristart(value, "Basic ", &p) &&
    state->auth_type <= HTTP_AUTH_BASIC) {
    state->auth_type = HTTP_AUTH_BASIC;


Do request for 401 again:
static int http_open_cnx(URLContext *h, AVDictionary **options)
{
...........................
if (s->http_code == 401) {
    if ((cur_auth_type == HTTP_AUTH_NONE || s->auth_state.stale) &&
        s->auth_state.auth_type != HTTP_AUTH_NONE && attempts < 4) {
        ffurl_closep(&s->hd);
        goto redo;

Could you please take a look and tell weather it is a bug or not ?

How to reproduce:

1. Create authorized location in nginx with config from attachment
2. Generate HLS playlist:

   ./ffmpeg.exe -re -stream_loop -1 -i /d/Video_samples/Video_sample_waves_1280x720.ts -vcodec libx264 -g 25 -acodec aac -strict -2 -f hls  -hls_time 10 -hls_list_size 10 -hls_flags delete_segments -hls_segment_filename /d/Program\ Files/nginx-1.14.0/hls/segment%03d.ts /d/Program\ Files/nginx-1.14.0/hls/test.m3u8
   

3. Download stream using ffmpeg:

   /d/Work/0_tmp/ffmpeg-4.3.2-2021-02-02-full_build/bin/ffmpeg.exe -report -loglevel trace -auth_type basic -i http://kraken:123456@192.168.0.12:8008/hls/test.m3u8 output.ts
   

4. Look at access.log from nginx and find several 401 error:

   192.168.0.12 - - [16/Feb/2021:14:50:22 +0300] "GET /hls/segment013.ts HTTP/1.1" 401 195 "-" "Lavf/58.45.100"
   192.168.0.12 - - [16/Feb/2021:14:50:22 +0300] "GET /hls/segment014.ts HTTP/1.1" 401 195 "-" "Lavf/58.45.100"
   

[sergey]

nginx authorization location

[sergey]

Last available build of ffmpeg is used:

anufriev.s@ANUFRIEV-S-01 MINGW64 /d/Work/Bug_364j
$ /d/Work/0_tmp/ffmpeg-4.3.2-2021-02-02-full_build/bin/ffmpeg.exe
ffmpeg version 4.3.2-2021-02-02-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10.2.0 (Rev6, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

[Carl Eugen Hoyos]

To make this a valid ticket please test current FFmpeg git head - nothing else is supported here - and provide the command line you tested together with the complete, uncut console output.

[sergey]

unauthorized access to nginx

[sergey]

Replying to cehoyos:

To make this a valid ticket please test current FFmpeg git head - nothing else is supported here - and provide the command line you tested together with the complete, uncut console output.

Hi,

Thank you for your reply.
I added full uncut log for ffmpeg HLS downloading with command line.

But I'm not able to build ffmpeg from git head under windows, because of weird errors during build. Anyway I can download the latest available official build from official site (see comment:1 above) and use it. From my point of view it is the latest I can use without any affort. Is it OK for ticket ?
Or maybe I should change version of ffmpeg in ticket ?