http cookies not set when using proxy

[Vineet Goel]

Summary of the bug: when -http_proxy option is used, ffmpeg doesn't honour "Set-Cookie:" header and so doesn't send "Cookie:" headers in subsequent request
How to reproduce:

ffprobe started on 2017-04-21 at 05:17:05
Report written to "ffprobe-20170421-051705.log"
Command line:
ffprobe -report http://cdn.rapticonsulting.in/gmtlive.php -http_proxy http://proxyuser:ffauth@144.217.86.183:8080
ffprobe version N-85595-g0f05f2c Copyright (c) 2007-2017 the FFmpeg developers
  built with gcc 4.9.2 (Debian 4.9.2-10)
  configuration: --prefix=/home/developer/ffmpeg_build --pkg-config-flags=--static --extra-cflags='-I/home/developer/ffmpeg_build/include -static' --extra-ldflags='-L/home/developer/ffmpeg_build/lib -static' --bindir=/home/developer/bin --disable-ffserver --disable-ffplay --disable-doc --enable-gray --enable-libx264 --enable-libmp3lame --enable-libfontconfig --enable-libfreetype --enable-gpl --enable-version3
  libavutil      55. 61.100 / 55. 61.100
  libavcodec     57. 93.100 / 57. 93.100
  libavformat    57. 72.101 / 57. 72.101
  libavdevice    57.  7.100 / 57.  7.100
  libavfilter     6. 84.101 /  6. 84.101
  libswscale      4.  7.101 /  4.  7.101
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100
[http @ 0x31d5360] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[http @ 0x31d5360] request: GET http://cdn.rapticonsulting.in:80/gmtlive.php HTTP/1.1
User-Agent: Lavf/57.72.101
Accept: */*
Range: bytes=0-
Connection: close
Host: cdn.rapticonsulting.in
Icy-MetaData: 1


[http @ 0x31d5360] request: GET http://cdn.rapticonsulting.in:80/gmtlive.php HTTP/1.1
User-Agent: Lavf/57.72.101
Accept: */*
Range: bytes=0-
Connection: close
Host: cdn.rapticonsulting.in
Icy-MetaData: 1
Proxy-Authorization: Basic cHJveHl1c2VyOmZmYXV0aA==


[http @ 0x31d5360] request: GET http://cdn.rapticonsulting.in:80/live/gmtlive/playlist.m3u8 HTTP/1.1
User-Agent: Lavf/57.72.101
Accept: */*
Range: bytes=0-
Connection: close
Host: cdn.rapticonsulting.in
Icy-MetaData: 1
Proxy-Authorization: Basic cHJveHl1c2VyOmZmYXV0aA==


[http @ 0x31d5360] HTTP error 403 Forbidden
http://cdn.rapticonsulting.in/gmtlive.php: Server returned 403 Forbidden (access denied)

If you try the same command without using proxy, i.e.,
% ffprobe -loglevel debug ​http://cdn.rapticonsulting.in/gmtlive.php

It accepts the cookies from the first request and sends them in subsequent request.

Also, I checked both the scenario with tcpdump, and Set-Cookie header is received in both cases on request to initial url.
I also checked the proxy scenario using curl,

% curl --verbose --location ​http://cdn.rapticonsulting.in/gmtlive.php --cookie /tmp/cookies.txt --cookie-jar /tmp/cookies.txt --proxy 144.217.86.183:8080 --proxy-user proxyuser:ffauth

and curl accepts the cookie and sends in them subsequent request.

and I did notice one difference. ffmpeg/ffprobe adds port number to domain part of url in proxy request whereas curl doesn't. like this,

with ffmpeg,
GET ​http://cdn.rapticonsulting.in:80/gmtlive.php HTTP/1.1
Host: cdn.rapticonsulting.in
...

with curl,
GET ​http://cdn.rapticonsulting.in/gmtlive.php HTTP/1.1
Host: cdn.rapticonsulting.in
......

Could this change in url be affecting cookie handling?

fyi, the cookies are actually for cloudfront secure access. and due to cookie not being set, access is denied on subsequent request. stream url given here is just for demo purpose to highlight this issue and for testing.
Actual deployment will have even the first request secured with token.

[Carl Eugen Hoyos]

Do you think this is a regression?

[Vineet Goel]

I don't think so

[Vineet Goel]

I looked into the source code (with my limited understanding of c), and issue seems to be how the "path" (GET line) is supposed to be different with proxy and non proxy request.

whereas, the get_cookies() function (in libavformat/http.c) doesn't take this into account while comparing path of stored cookie and request uri. and so it prevents sending the cookies stored from previous request in subsequent request.

shall I try a patch?

thanks,

[Vineet Goel]

i found the fix. infact the code already maintains local_path as well path variables. which are same in case of non-proxy request but differ in case of proxy request. and for getting cookies, local_path shall be used always.
so i made following changes,

libavformat/http.c, line: 1097

replaced,
if (!get_cookies(s, &cookies, path, hoststr) && cookies) {
with
if (!get_cookies(s, &cookies, local_path, hoststr) && cookies) {

and cookies work now with proxy as well.
patch file attached. can this be accepted as official fix ?

thank you,

[Vineet Goel]

fix to correctly send cookies when using proxy

[Carl Eugen Hoyos]

Please create a patch with git format-patch and send it to the development mailing list.

[Vineet Goel]

sent