Opened 6 months ago

Last modified 6 months ago

#6384 new defect

Bug in URL decoding in Windows version

Reported by: fa1rid Owned by:
Priority: normal Component: ffmpeg
Version: unspecified Keywords: url
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

I try to download a video using Ubuntu version and it's successful but when I use any windows version it gives me HTTP error 403 Forbidden

The problem is only with https links.

This is the command and log from Windows 8.1
I tried many versions of ffmpeg on windows but same problem.

ffmpeg -user_agent "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit?/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36" -i "https://fbchdvod-f.akamaihd.net/i/Fox.com/209/75/PrisonBreak_1AZM06_9373875879,73_mp4_video_1280x720_2500000_primary_audio_8,72_mp4_video_1280x720_1800000_primary_audio_7,71_mp4_video_1024x576_1300000_primary_audio_6,70_mp4_video_640x360_800000_primary_audio_5,69_mp4_video_480x270_500000_primary_audio_4,68_mp4_video_480x270_300000_primary_audio_3,80_mp4_video_400x0_225000_primary_audio_2,78_mp4_video_400x0_150000_primary_audio_1,.mp4.csmil/index_0_av.m3u8?null=0&id=AgBKyTkkF0P8FXYfE1nEwR%2f%2fV9FBSV6q1dFTtB1O6Zt0w6F0DEZvERLpnEcJXZTn0WXq5w8dJPb1tQ%3d%3d&hdntl=exp=1494511862~acl=%2fi%2fFox.com%2f209%2f75%2fPrisonBreak_1AZM06_9373875879*~data=hdntl~hmac=f7564e58f026f8dfed7e35a8e5fd32075d574124a71cdd3a3fee9eedd86f80d3" -c copy -bsf:a aac_adtstoasc video.mp4

ffmpeg version N-85750-ga75ef15 Copyright (c) 2000-2017 the FFmpeg developers

built with gcc 6.3.0 (GCC)
configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --e

nable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --
enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv
--enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-li
bfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug -
-enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enabl
e-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-li
bsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolam
e --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx

--enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable

-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-zlib

libavutil 55. 61.100 / 55. 61.100
libavcodec 57. 93.100 / 57. 93.100
libavformat 57. 72.101 / 57. 72.101
libavdevice 57. 7.100 / 57. 7.100
libavfilter 6. 88.100 / 6. 88.100
libswscale 4. 7.101 / 4. 7.101
libswresample 2. 8.100 / 2. 8.100
libpostproc 54. 6.100 / 54. 6.100

[https @ 0000000001cd37c0] HTTP error 403 Forbidden
https://fbchdvod-f.akamaihd.net/i/Fox.com/209/75/PrisonBreak_1AZM06_9373875879,7
3_mp4_video_1280x720_2500000_primary_audio_8,72_mp4_video_1280x720_1800000_prima
ry_audio_7,71_mp4_video_1024x576_1300000_primary_audio_6,70_mp4_video_640x360_80
0000_primary_audio_5,69_mp4_video_480x270_500000_primary_audio_4,68_mp4_video_48
0x270_300000_primary_audio_3,80_mp4_video_400x0_225000_primary_audio_2,78_mp4_vi
deo_400x0_150000_primary_audio_1,.mp4.csmil/index_0_av.m3u8?null=0&id=AgBKyTkkF0
P8FXYfE1nEwRffV9FBSV6q1dFTtB1O6Zt0w6F0DEZvERLpnEcJXZTn0WXq5w8dJPb1tQdd&hdntl=exp
=1494511862~acl=fifFox.comf209f75fPrisonBreak_1AZM06_9373875879*~data=hdntl~hmac
=f7564e58f026f8dfed7e35a8e5fd32075d574124a71cdd3a3fee9eedd86f80d3: Server return
ed 403 Forbidden (access denied)

This is the same command on Ubuntu but with success:

ffmpeg -user_agent "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit?/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36" -i "https://fbchdvod-f.akamaihd.net/i/Fox.com/209/75/PrisonBreak_1AZM06_9373875879,73_mp4_video_1280x720_2500000_primary_audio_8,72_mp4_video_1280x720_1800000_primary_audio_7,71_mp4_video_1024x576_1300000_primary_audio_6,70_mp4_video_640x360_800000_primary_audio_5,69_mp4_video_480x270_500000_primary_audio_4,68_mp4_video_480x270_300000_primary_audio_3,80_mp4_video_400x0_225000_primary_audio_2,78_mp4_video_400x0_150000_primary_audio_1,.mp4.csmil/index_0_av.m3u8?null=0&id=AgBKyTkkF0P8FXYfE1nEwR%2f%2fV9FBSV6q1dFTtB1O6Zt0w6F0DEZvERLpnEcJXZTn0WXq5w8dJPb1tQ%3d%3d&hdntl=exp=1494511862~acl=%2fi%2fFox.com%2f209%2f75%2fPrisonBreak_1AZM06_9373875879*~data=hdntl~hmac=f7564e58f026f8dfed7e35a8e5fd32075d574124a71cdd3a3fee9eedd86f80d3" -c copy -bsf:a aac_adtstoasc video.mp4
ffmpeg version 3.3 Copyright (c) 2000-2017 the FFmpeg developers

built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
configuration: --extra-libs=-ldl --prefix=/opt/ffmpeg --mandir=/usr/share/man --enable-avresample --disable-debug --enable-nonfree --enable-gpl --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --disable-decoder=amrnb --disable-decoder=amrwb --enable-libpulse --enable-libfreetype --enable-gnutls --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-libvorbis --enable-libmp3lame --enable-libopus --enable-libvpx --enable-libspeex --enable-libass --enable-avisynth --enable-libsoxr --enable-libxvid --enable-libvidstab --enable-libwavpack --enable-nvenc
libavutil 55. 58.100 / 55. 58.100
libavcodec 57. 89.100 / 57. 89.100
libavformat 57. 71.100 / 57. 71.100
libavdevice 57. 6.100 / 57. 6.100
libavfilter 6. 82.100 / 6. 82.100
libavresample 3. 5. 0 / 3. 5. 0
libswscale 4. 6.100 / 4. 6.100
libswresample 2. 7.100 / 2. 7.100
libpostproc 54. 5.100 / 54. 5.100

[NULL @ 0x3198d40] non-existing SPS 0 referenced in buffering period
[NULL @ 0x3198d40] SPS unavailable in decode_picture_timing
[h264 @ 0x31a6420] non-existing SPS 0 referenced in buffering period
[h264 @ 0x31a6420] SPS unavailable in decode_picture_timing
Input #0, hls,applehttp, from 'https://fbchdvod-f.akamaihd.net/i/Fox.com/209/75/PrisonBreak_1AZM06_9373875879,73_mp4_video_1280x720_2500000_primary_audio_8,72_mp4_video_1280x720_1800000_primary_audio_7,71_mp4_video_1024x576_1300000_primary_audio_6,70_mp4_video_640x360_800000_primary_audio_5,69_mp4_video_480x270_500000_primary_audio_4,68_mp4_video_480x270_300000_primary_audio_3,80_mp4_video_400x0_225000_primary_audio_2,78_mp4_video_400x0_150000_primary_audio_1,.mp4.csmil/index_0_av.m3u8?null=0&id=AgBKyTkkF0P8FXYfE1nEwR%2f%2fV9FBSV6q1dFTtB1O6Zt0w6F0DEZvERLpnEcJXZTn0WXq5w8dJPb1tQ%3d%3d&hdntl=exp=1494511862~acl=%2fi%2fFox.com%2f209%2f75%2fPrisonBreak_1AZM06_9373875879*~data=hdntl~hmac=f7564e58f026f8dfed7e35a8e5fd32075d574124a71cdd3a3fee9eedd86f80d3':

Duration: 00:42:49.91, start: 0.100511, bitrate: 0 kb/s
Program 0

Metadata:

variant_bitrate : 0

Stream #0:0: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 23.98 fps, 23.98 tbr, 90k tbn, 47.95 tbc
Metadata:

variant_bitrate : 0

Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp
Metadata:

variant_bitrate : 0

Output #0, mp4, to 'video.mp4':

Metadata:

encoder : Lavf57.71.100
Stream #0:0: Video: h264 (Main) ([33][0][0][0] / 0x0021), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 23.98 fps, 23.98 tbr, 90k tbn, 90k tbc
Metadata:

variant_bitrate : 0

Stream #0:1: Audio: aac (LC) ([64][0][0][0] / 0x0040), 44100 Hz, stereo, fltp
Metadata:

variant_bitrate : 0

Stream mapping:

Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)

Press [q] to stop, ? for help
frame= 6954 fps=759 q=-1.0 Lsize= 93261kB time=00:04:49.99 bitrate=2634.5kbits/s speed=31.6x
video:88510kB audio:4447kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.327113%
Exiting normally, received signal 2.

Change History (8)

comment:1 Changed 6 months ago by cehoyos

  • Keywords Windows version removed
  • Priority changed from important to normal

Does the link still work on Ubuntu?
Is it possible that the link simply timed out by the time you tested on Windows?

comment:3 Changed 6 months ago by fa1rid

I found what's the problem.
The url i'm giving to ffmpeg is url-encoded

When I traced the url being requested I found that ffmpeg has removed all %(x) values.

So the url is :

https://fbchdvod-f.akamaihd.net/i/Fox.com/209/75/PrisonBreak_1AZM06_9373875879,73_mp4_video_1280x720_2500000_primary_audio_8,72_mp4_video_1280x720_1800000_primary_audio_7,71_mp4_video_1024x576_1300000_primary_audio_6,70_mp4_video_640x360_800000_primary_audio_5,69_mp4_video_480x270_500000_primary_audio_4,68_mp4_video_480x270_300000_primary_audio_3,80_mp4_video_400x0_225000_primary_audio_2,78_mp4_video_400x0_150000_primary_audio_1,.mp4.csmil/index_0_av.m3u8?null=0&id=AgCbXv4PuBy89LYzFlmxZPELGgrH%2fPRdFYxaqH27rH7FkdNqUlEyMgOgg6MoSjP9SnzNqH7dIt1roA%3d%3d&hdntl=exp=1494713654~acl=%2fi%2fFox.com%2f209%2f75%2fPrisonBreak_1AZM06_9373875879*~data=hdntl~hmac=5bc9f51ae91f63e967b729823c176b6532fcf09eae8e98355ae4f59d830cd9a0

ffmpeg requested this link instead:

https://fbchdvod-f.akamaihd.net/i/Fox.com/209/75/PrisonBreak_1AZM06_9373875879,73_mp4_video_1280x720_2500000_primary_audio_8,72_mp4_video_1280x720_1800000_primary_audio_7,71_mp4_video_1024x576_1300000_primary_audio_6,70_mp4_video_640x360_800000_primary_audio_5,69_mp4_video_480x270_500000_primary_audio_4,68_mp4_video_480x270_300000_primary_audio_3,80_mp4_video_400x0_225000_primary_audio_2,78_mp4_video_400x0_150000_primary_audio_1,.mp4.csmil/index_0_av.m3u8?null=0&id=AgCbXv4PuBy89LYzFlmxZPELGgrHfPRdFYxaqH27rH7FkdNqUlEyMgOgg6MoSjP9SnzNqH7dIt1roAdd&hdntl=exp=1494713654~acl=fifFox.comf209f75fPrisonBreak_1AZM06_9373875879*~data=hdntl~hmac=5bc9f51ae91f63e967b729823c176b6532fcf09eae8e98355ae4f59d830cd9a0

comment:4 Changed 6 months ago by fa1rid

  • Analyzed by developer set
  • Component changed from undetermined to ffmpeg
  • Keywords url added; https removed

comment:5 Changed 6 months ago by fa1rid

  • Summary changed from Windows version has problem with https vs Ubuntu version not to Bug in URL decoding in Windows version

comment:6 Changed 6 months ago by fa1rid

regarding why ubuntu showed me same error is that because the new link works only for the new user-agent which became Chrome/58.0.3029.110 instead of Chrome/57.0.2987.133
So the problem is now confirmed to be URL decoding issue.

comment:7 Changed 6 months ago by heleppkes

  • Analyzed by developer unset

URL parsing works fine for me, ffmpeg receives the URL just like I input it. Make sure to properly quote the string.

comment:8 Changed 6 months ago by fa1rid

It's not working fine for me. Have you successfully downloaded the file then? Where's your proof that it requests the same URL? I have proof from fiddler that it changed it as I said. Maybe only the parameters after .m3u8?null=.... which considered web forms is not being decoded correctly. Please double check and notice the parameters.

Note: See TracTickets for help on using tickets.