Opened 13 years ago

Closed 8 years ago

#1291 closed defect (fixed)

The custom user-agent option is ignored after the initial connection when using HTTP Live Streaming (HLS)

Reported by: Pan. Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: hls
Cc: blappv@gmail.com Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Hello.

The "-user-agent" option is ignored after the initial connection to an HLS server.

When running this command line:
ffmpeg -user-agent myuseragent ​http://someserver/playlist.m3u8 -c copy -

I get this network dump for the initial connection to someserver:

GET /playlist.m3u8 HTTP/1.1.
User-Agent: myuseragent
...

But in the (automated) consecutive connections (after the initial .ts segments are consumed) I see this:

GET /playlist.m3u8 HTTP/1.1.
User-Agent: Lavf54.3.100.
...

Which means that the custom user-agent option is somehow ignored after the initial connection to the server.

Version: ffmpeg version N-40092-g3bbf3f7
OS: Windows (MinGW)

Change History (6)

comment:1 by Carl Eugen Hoyos, 12 years ago

Component: undetermined → avformat
Keywords: hls added; user-agent HLS HTTP Live Streaming removed

Please add complete, uncut console output.

in reply to:  1 comment:2 by Pan., 12 years ago

Replying to cehoyos:

Please add complete, uncut console output.

This snippet:

GET /playlist.m3u8 HTTP/1.1.
User-Agent: Lavf54.3.100.
...

is observed via the network diagnostics tool "tcpdump" and not the console.

The console does not output anything regarding the user-agent at all.
But here it is:

ffmpeg version N-40092-g3bbf3f7 Copyright (c) 2000-2012 the FFmpeg developers

built on Apr 24 2012 00:55:38 with gcc 4.6.3
configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-runtime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libass --enable-libcelt --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-libnut --enable-libopenjpeg --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libutvideo --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
libavutil 51. 47.100 / 51. 47.100
libavcodec 54. 15.100 / 54. 15.100
libavformat 54. 3.100 / 54. 3.100
libavdevice 53. 4.100 / 53. 4.100
libavfilter 2. 72.100 / 2. 72.100
libswscale 2. 1.100 / 2. 1.100
libswresample 0. 11.100 / 0. 11.100
libpostproc 52. 0.100 / 52. 0.100

[hls,applehttp @ 0225d9c0] Format hls,applehttp probed with size=2048 and score=100
Format mpegts probed with size=2048 and score=100
[mpegts @ 023482e0] stream=0 stream_type=1b pid=21 prog_reg_desc=
[mpegts @ 023482e0] stream=1 stream_type=f pid=24 prog_reg_desc=
[hls,applehttp @ 0225d9c0] File position before avformat_find_stream_info() is 1959
[mpegts @ 023482e0] Continuity check failed for pid 0 expected 1 got 3
[NULL @ 02347780] Current profile doesn't provide more RBSP data in PPS, skipping
[hls,applehttp @ 0225d9c0] All info found
[hls,applehttp @ 0225d9c0] File position after avformat_find_stream_info() is 1959
Input #0, hls,applehttp, from '​http://someserver/playlist.m3u8':

Duration: N/A, start: 48533.090044, bitrate: N/A

Stream #0:0, 21, 1/90000: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360, 1/50, 25 tbr, 90k tbn, 50 tbc
Stream #0:1, 18, 1/90000: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, s16

Error parsing AAC extradata, unable to determine samplerate.
Output #0, matroska, to 'nul':

Metadata:

encoder : Lavf54.3.100
Stream #0:0, 0, 1/1000: Video: h264 ([27][0][0][0] / 0x001B), yuv420p, 640x360, 1/90000, q=2-31, 1k tbn, 90k tbc
Stream #0:1, 0, 1/1000: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo

Stream mapping:

Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)

Press [q] to stop, ? for help
...

comment:3 by Carl Eugen Hoyos, 12 years ago

Resolution: → fixed
Status: new → closed
Version: unspecified → git-master

Fixed by Micah Galizia.

comment:4 by blappv, 12 years ago

Cc: blappv@gmail.com added
Resolution: fixed
Status: closed → reopened

I passed user-agent "User-Agent: My MX Player/1.0" within other custom headers (cookie) with url "​http://devimages.apple.com/iphone/samples/bipbop/gear1/prog_index.m3u8":

The first request was sent as expected:

GET /devimages.apple.com/iphone/samples/bipbop/gear1/prog_index.m3u8 HTTP/1.1
Accept: */*
Connection: close
Host: devimages.apple.com
User-Agent: My MX Player/1.0
Cookie: mobileuser=blapp

But the subsequence requests ignored the custom headers:

GET /devimages.apple.com/iphone/samples/bipbop/gear1/fileSequence0.ts HTTP/1.1
User-Agent: Lavf54.63.102
Accept: */*
Connection: close
Host: devimages.apple.com


The original issue was reported at ​https://groups.google.com/forum/?fromgroups=#!topic/mx-videoplayer/w4rpkK66zMc

comment:5 by Carl Eugen Hoyos, 11 years ago

Ticket #2485 contains another example of a not-forwarded option.

comment:6 by Steven Liu, 8 years ago

Resolution: → fixed
Status: reopened → closed

New version has fixed this problem:

localhost:commit StevenLiu$ ffmpeg -v trace -user-agent "Steven Liu" -i "http://devimages.apple.com/iphone/samples/bipbop/gear1/prog_index.m3u8"
ffmpeg version N-82254-g75a1311 Copyright (c) 2000-2016 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
  configuration: --enable-libass --enable-opengl --enable-libx264 --enable-libmp3lame --enable-gpl --enable-nonfree --prefix=/usr/local --enable-libopencv --enable-libtesseract --enable-libspeex --enable-libfreetype --enable-libfontconfig --enable-libfdk-aac
  libavutil      55. 35.100 / 55. 35.100
  libavcodec     57. 66.101 / 57. 66.101
  libavformat    57. 57.100 / 57. 57.100
  libavdevice    57.  2.100 / 57.  2.100
  libavfilter     6. 66.100 /  6. 66.100
  libswscale      4.  3.100 /  4.  3.100
  libswresample   2.  4.100 /  2.  4.100
  libpostproc    54.  2.100 / 54.  2.100
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument 'trace'.
Reading option '-user-agent' ... matched as AVOption 'user-agent' with argument 'Steven Liu'.
Reading option '-i' ... matched as input file with argument 'http://devimages.apple.com/iphone/samples/bipbop/gear1/prog_index.m3u8'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument trace.
Successfully parsed a group of options.
Parsing a group of options: input file http://devimages.apple.com/iphone/samples/bipbop/gear1/prog_index.m3u8.
Successfully parsed a group of options.
Opening an input file: http://devimages.apple.com/iphone/samples/bipbop/gear1/prog_index.m3u8.
[http @ 0x7fa75a8030c0] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[http @ 0x7fa75a006800] the user-agent option is deprecated, please use user_agent option
[http @ 0x7fa75a8030c0] request: GET /iphone/samples/bipbop/gear1/prog_index.m3u8 HTTP/1.1
User-Agent: Steven Liu
Accept: */*
Range: bytes=0-
Connection: close
Host: devimages.apple.com
Icy-MetaData: 1


[http @ 0x7fa75a8030c0] header='HTTP/1.1 206 Partial Content'
[http @ 0x7fa75a8030c0] http_code=206
[http @ 0x7fa75a8030c0] header='Server: Apache'
[http @ 0x7fa75a8030c0] header='ETag: "50117c8233644c19b5ab49551b72507f:1239907352"'
[http @ 0x7fa75a8030c0] header='Last-Modified: Thu, 16 Apr 2009 18:42:32 GMT'
[http @ 0x7fa75a8030c0] header='Accept-Ranges: bytes'
[http @ 0x7fa75a8030c0] header='Content-Type: audio/x-mpegurl'
[http @ 0x7fa75a8030c0] header='Date: Fri, 17 Feb 2017 16:01:10 GMT'
[http @ 0x7fa75a8030c0] header='Content-Range: bytes 0-7018/7019'
[http @ 0x7fa75a8030c0] header='Content-Length: 7019'
[http @ 0x7fa75a8030c0] header='Connection: close'
[http @ 0x7fa75a8030c0] header=''
Probing hls,applehttp score:100 size:2048
[hls,applehttp @ 0x7fa75a006200] Format hls,applehttp probed with size=2048 and score=100
[hls,applehttp @ 0x7fa75a006200] new_program: id=0x0000
[hls,applehttp @ 0x7fa75a006200] HLS request for url 'http://devimages.apple.com/iphone/samples/bipbop/gear1/fileSequence0.ts', offset 0, playlist 0
[http @ 0x7fa75980ac00] the user-agent option is deprecated, please use user_agent option
[http @ 0x7fa758c13280] request: GET /iphone/samples/bipbop/gear1/fileSequence0.ts HTTP/1.1
User-Agent: Steven Liu
Accept: */*
Connection: close
Host: devimages.apple.com
Icy-MetaData: 1


[http @ 0x7fa758c13280] header='HTTP/1.1 200 OK'
[http @ 0x7fa758c13280] http_code=200
[http @ 0x7fa758c13280] header='Server: Apache'
[http @ 0x7fa758c13280] header='ETag: "90e466ad7f45ea3bd52732124f1fa675:1239907291"'
[http @ 0x7fa758c13280] header='Last-Modified: Thu, 16 Apr 2009 18:41:31 GMT'
[http @ 0x7fa758c13280] header='Accept-Ranges: bytes'
[http @ 0x7fa758c13280] header='Content-Length: 250228'
[http @ 0x7fa758c13280] header='Content-Type: video/mp2t'
[http @ 0x7fa758c13280] header='Date: Fri, 17 Feb 2017 16:01:11 GMT'
[http @ 0x7fa758c13280] header='Connection: close'
[http @ 0x7fa758c13280] header=''
Probing mp3 score:1 size:2048
Probing mpegts score:50 size:2048
Format mpegts probed with size=2048 and score=50
score: 44, dvhs_score: -3, fec_score: -3
[mpegts @ 0x7fa759801000] Filter: pid=0x11 type=1
[mpegts @ 0x7fa759801000] Filter: pid=0x0 type=1
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] sid=0x1 pid=0x100
[mpegts @ 0x7fa759801000] new_program: id=0x0001
[mpegts @ 0x7fa759801000] Filter: pid=0x100 type=1
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] sid=0x1 sec_num=0/0 version=0 tid=2
[mpegts @ 0x7fa759801000] pcr_pid=0x102
[mpegts @ 0x7fa759801000] Filter: pid=0x101 type=0
[mpegts @ 0x7fa759801000] stream=0 stream_type=f pid=101 prog_reg_desc=
[mpegts @ 0x7fa759801000] Filter: pid=0x102 type=0
[mpegts @ 0x7fa759801000] stream=1 stream_type=1b pid=102 prog_reg_desc=
[mpegts @ 0x7fa759801000] tuning done
[hls,applehttp @ 0x7fa75a006200] Before avformat_find_stream_info() pos: 7019 bytes read:7019 seeks:0 nb_streams:2
[mpegts @ 0x7fa759801000] Skipping after seek
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] sid=0x1 pid=0x100
[mpegts @ 0x7fa759801000] new_program: id=0x0001
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] sid=0x1 sec_num=0/0 version=0 tid=2
[mpegts @ 0x7fa759801000] pcr_pid=0x102
[mpegts @ 0x7fa759801000] All programs have pmt, headers found
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
    Last message repeated 1 times
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[h264 @ 0x7fa759025c00] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x7fa759025c00] nal_unit_type: 7, nal_ref_idc: 1
[h264 @ 0x7fa759025c00] nal_unit_type: 8, nal_ref_idc: 1
[h264 @ 0x7fa759025c00] nal_unit_type: 6, nal_ref_idc: 0
    Last message repeated 1 times
[h264 @ 0x7fa759025c00] nal_unit_type: 5, nal_ref_idc: 1
    Last message repeated 1 times
[h264 @ 0x7fa759025c00] Reinit context to 192x144, pix_fmt: yuv420p
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[h264 @ 0x7fa759025c00] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x7fa759025c00] nal_unit_type: 6, nal_ref_idc: 0
[h264 @ 0x7fa759025c00] nal_unit_type: 1, nal_ref_idc: 1
    Last message repeated 1 times
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
    Last message repeated 1 times
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[h264 @ 0x7fa759025c00] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x7fa759025c00] nal_unit_type: 6, nal_ref_idc: 0
[h264 @ 0x7fa759025c00] nal_unit_type: 1, nal_ref_idc: 1
    Last message repeated 1 times
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[h264 @ 0x7fa759025c00] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x7fa759025c00] nal_unit_type: 6, nal_ref_idc: 0
[h264 @ 0x7fa759025c00] nal_unit_type: 1, nal_ref_idc: 1
    Last message repeated 1 times
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[h264 @ 0x7fa759025c00] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x7fa759025c00] nal_unit_type: 6, nal_ref_idc: 0
[h264 @ 0x7fa759025c00] nal_unit_type: 1, nal_ref_idc: 1
    Last message repeated 1 times
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[h264 @ 0x7fa759025c00] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x7fa759025c00] nal_unit_type: 6, nal_ref_idc: 0
[h264 @ 0x7fa759025c00] nal_unit_type: 1, nal_ref_idc: 1
    Last message repeated 1 times
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[h264 @ 0x7fa759025c00] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x7fa759025c00] nal_unit_type: 6, nal_ref_idc: 0
[h264 @ 0x7fa759025c00] nal_unit_type: 1, nal_ref_idc: 1
    Last message repeated 1 times
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
    Last message repeated 1 times
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
[mpegts @ 0x7fa759801000] PAT:
[mpegts @ 0x7fa759801000] PMT: len 26
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[mpegts @ 0x7fa759801000] pid=101 pes_code=0x1c0
    Last message repeated 1 times
[mpegts @ 0x7fa759801000] pid=102 pes_code=0x1e0
[hls,applehttp @ 0x7fa75a006200] All info found
[hls,applehttp @ 0x7fa75a006200] Setting avg frame rate based on r frame rate
[hls,applehttp @ 0x7fa75a006200] stream 0: start_time: 10.000 duration: -102481911520608.625
[hls,applehttp @ 0x7fa75a006200] stream 1: start_time: 10.000 duration: -102481911520608.625
[hls,applehttp @ 0x7fa75a006200] format: start_time: 10.000 duration: 1801.000 bitrate=0 kb/s
[hls,applehttp @ 0x7fa75a006200] After avformat_find_stream_info() pos: 7019 bytes read:7019 seeks:0 frames:53
Input #0, hls,applehttp, from 'http://devimages.apple.com/iphone/samples/bipbop/gear1/prog_index.m3u8':
  Duration: 00:30:01.00, start: 10.000000, bitrate: 0 kb/s
  Program 0
    Metadata:
      variant_bitrate : 0
    Stream #0:0, 32, 1/90000: Audio: aac (LC) ([15][0][0][0] / 0x000F), 22050 Hz, mono, fltp
    Metadata:
      variant_bitrate : 0
    Stream #0:1, 21, 1/90000: Video: h264 (Constrained Baseline), 1 reference frame ([27][0][0][0] / 0x001B), yuv420p(tv, smpte170m/smpte170m/bt709, topleft), 192x144, 0/1, 14.99 fps, 14.99 tbr, 90k tbn, 180k tbc
    Metadata:
      variant_bitrate : 0
Successfully opened the file.
At least one output file must be specified
[AVIOContext @ 0x7fa758c13500] Statistics: 65538 bytes read, 0 seeks
[AVIOContext @ 0x7fa75a803ba0] Statistics: 7019 bytes read, 0 seeks
localhost:commit StevenLiu$
Note: See TracTickets for help on using tickets.