Opened 3 months ago

Last modified 3 months ago

#6338 new defect

http cookies not set when using proxy

Reported by: vineet156 Owned by:
Priority: normal Component: undetermined
Version: git-master Keywords: http, hls, proxy
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug: when -http_proxy option is used, ffmpeg doesn't honour "Set-Cookie:" header and so doesn't send "Cookie:" headers in subsequent request
How to reproduce:

ffprobe started on 2017-04-21 at 05:17:05
Report written to "ffprobe-20170421-051705.log"
Command line:
ffprobe -report http://cdn.rapticonsulting.in/gmtlive.php -http_proxy http://proxyuser:ffauth@144.217.86.183:8080
ffprobe version N-85595-g0f05f2c Copyright (c) 2007-2017 the FFmpeg developers
  built with gcc 4.9.2 (Debian 4.9.2-10)
  configuration: --prefix=/home/developer/ffmpeg_build --pkg-config-flags=--static --extra-cflags='-I/home/developer/ffmpeg_build/include -static' --extra-ldflags='-L/home/developer/ffmpeg_build/lib -static' --bindir=/home/developer/bin --disable-ffserver --disable-ffplay --disable-doc --enable-gray --enable-libx264 --enable-libmp3lame --enable-libfontconfig --enable-libfreetype --enable-gpl --enable-version3
  libavutil      55. 61.100 / 55. 61.100
  libavcodec     57. 93.100 / 57. 93.100
  libavformat    57. 72.101 / 57. 72.101
  libavdevice    57.  7.100 / 57.  7.100
  libavfilter     6. 84.101 /  6. 84.101
  libswscale      4.  7.101 /  4.  7.101
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100
[http @ 0x31d5360] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[http @ 0x31d5360] request: GET http://cdn.rapticonsulting.in:80/gmtlive.php HTTP/1.1
User-Agent: Lavf/57.72.101
Accept: */*
Range: bytes=0-
Connection: close
Host: cdn.rapticonsulting.in
Icy-MetaData: 1


[http @ 0x31d5360] request: GET http://cdn.rapticonsulting.in:80/gmtlive.php HTTP/1.1
User-Agent: Lavf/57.72.101
Accept: */*
Range: bytes=0-
Connection: close
Host: cdn.rapticonsulting.in
Icy-MetaData: 1
Proxy-Authorization: Basic cHJveHl1c2VyOmZmYXV0aA==


[http @ 0x31d5360] request: GET http://cdn.rapticonsulting.in:80/live/gmtlive/playlist.m3u8 HTTP/1.1
User-Agent: Lavf/57.72.101
Accept: */*
Range: bytes=0-
Connection: close
Host: cdn.rapticonsulting.in
Icy-MetaData: 1
Proxy-Authorization: Basic cHJveHl1c2VyOmZmYXV0aA==


[http @ 0x31d5360] HTTP error 403 Forbidden
http://cdn.rapticonsulting.in/gmtlive.php: Server returned 403 Forbidden (access denied)

If you try the same command without using proxy, i.e.,
% ffprobe -loglevel debug http://cdn.rapticonsulting.in/gmtlive.php

It accepts the cookies from the first request and sends them in subsequent request.

Also, I checked both the scenario with tcpdump, and Set-Cookie header is received in both cases on request to initial url.
I also checked the proxy scenario using curl,

% curl --verbose --location http://cdn.rapticonsulting.in/gmtlive.php --cookie /tmp/cookies.txt --cookie-jar /tmp/cookies.txt --proxy 144.217.86.183:8080 --proxy-user proxyuser:ffauth

and curl accepts the cookie and sends in them subsequent request.

and I did notice one difference. ffmpeg/ffprobe adds port number to domain part of url in proxy request whereas curl doesn't. like this,

with ffmpeg,
GET http://cdn.rapticonsulting.in:80/gmtlive.php HTTP/1.1
Host: cdn.rapticonsulting.in
...

with curl,
GET http://cdn.rapticonsulting.in/gmtlive.php HTTP/1.1
Host: cdn.rapticonsulting.in
......

Could this change in url be affecting cookie handling?

fyi, the cookies are actually for cloudfront secure access. and due to cookie not being set, access is denied on subsequent request. stream url given here is just for demo purpose to highlight this issue and for testing.
Actual deployment will have even the first request secured with token.

Attachments (1)

http_cookies_with_proxy.patch (594 bytes) - added by vineet156 3 months ago.
fix to correctly send cookies when using proxy

Download all attachments as: .zip

Change History (8)

comment:1 Changed 3 months ago by cehoyos

Do you think this is a regression?

comment:2 Changed 3 months ago by vineet156

I don't think so

comment:3 Changed 3 months ago by cehoyos

  • Priority changed from important to normal

comment:4 Changed 3 months ago by vineet156

I looked into the source code (with my limited understanding of c), and issue seems to be how the "path" (GET line) is supposed to be different with proxy and non proxy request.

whereas, the get_cookies() function (in libavformat/http.c) doesn't take this into account while comparing path of stored cookie and request uri. and so it prevents sending the cookies stored from previous request in subsequent request.

shall I try a patch?

thanks,

comment:5 Changed 3 months ago by vineet156

i found the fix. infact the code already maintains local_path as well path variables. which are same in case of non-proxy request but differ in case of proxy request. and for getting cookies, local_path shall be used always.
so i made following changes,

libavformat/http.c, line: 1097

replaced,
if (!get_cookies(s, &cookies, path, hoststr) && cookies) {
with
if (!get_cookies(s, &cookies, local_path, hoststr) && cookies) {

and cookies work now with proxy as well.
patch file attached. can this be accepted as official fix ?

thank you,

Changed 3 months ago by vineet156

fix to correctly send cookies when using proxy

comment:6 Changed 3 months ago by cehoyos

Please create a patch with git format-patch and send it to the development mailing list.

comment:7 Changed 3 months ago by vineet156

sent

Note: See TracTickets for help on using tickets.