Opened 5 years ago

Closed 4 years ago

#7660 closed defect (fixed)

Unable to play URLs with fragment identifier

Reported by: remitamine Owned by:
Priority: normal Component: avformat
Version: unspecified Keywords: http
Cc: Marton Balint Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:
FFmpeg does not handle URLs with fragment identifier, VLC plays the same URLs without a problem.
How to reproduce:

ffplay -v trace $(curl https://www.dailymotion.com/player/metadata/video/x6ykqgm | jq -r .qualities.auto[0].url)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11301    0 11301    0     0  17740      0 --:--:-- --:--:-- --:--:-- 17713
ffplay version n4.1 Copyright (c) 2003-2018 the FFmpeg developers
  built with gcc 8.2.1 (GCC) 20180831
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-nvdec --enable-nvenc --enable-omx --enable-shared --enable-version3
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Initialized opengl renderer.
[NULL @ 0x7efdb4000b80] Opening 'https://www.dailymotion.com/cdn/manifest/video/x6ykqgm.m3u8?sec=Hp1KzZeSUhvvwGdZmd4x5K8jkxgnOYC771Zyn8FZkJUN_AWj69qF7OeErQkRTW6D' for reading
[https @ 0x7efdb4001800] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[tcp @ 0x7efdb4003dc0] Original list of addresses:sq=    0B f=0/0   
[tcp @ 0x7efdb4003dc0] Address 195.8.215.136 port 443
[tcp @ 0x7efdb4003dc0] Interleaved list of addresses:
[tcp @ 0x7efdb4003dc0] Address 195.8.215.136 port 443
[tcp @ 0x7efdb4003dc0] Starting connection attempt to 195.8.215.136 port 443
[tcp @ 0x7efdb4003dc0] Successfully connected to 195.8.215.136 port 443
[https @ 0x7efdb4001800] request: GET /cdn/manifest/video/x6ykqgm.m3u8?sec=Hp1KzZeSUhvvwGdZmd4x5K8jkxgnOYC771Zyn8FZkJUN_AWj69qF7OeErQkRTW6D HTTP/1.1
User-Agent: Lavf/58.20.100
Accept: */*
Range: bytes=0-
Connection: close
Host: www.dailymotion.com
Icy-MetaData: 1


[https @ 0x7efdb4001800] header='HTTP/1.1 206 Partial Content'0/0   
[https @ 0x7efdb4001800] http_code=206
[https @ 0x7efdb4001800] header='Content-Security-Policy-Report-Only: default-src https: blob: wss: data: android-webview-video-poster: android-webview: chrome-extension: safari-extension: 'unsafe-eval' 'unsafe-inline'; report-uri https://www.dailymotion.com/content_security_policy_report_uri'
[https @ 0x7efdb4001800] header='Server: DMS/1.0.42'
[https @ 0x7efdb4001800] header='X-DM-BackNode: web-423.adm.dc3.dailymotion.com:80'
[https @ 0x7efdb4001800] header='Vary: X-DM-SSL,Accept-Encoding'
[https @ 0x7efdb4001800] header='Cache-Control: private, max-age=60'
[https @ 0x7efdb4001800] header='Content-Type: application/vnd.apple.mpegurl'
[https @ 0x7efdb4001800] header='Content-Security-Policy: upgrade-insecure-requests'
[https @ 0x7efdb4001800] header='Access-Control-Expose-Headers: X-DM-BackNode-Response-Time'
[https @ 0x7efdb4001800] header='Strict-Transport-Security: max-age=31708800; includeSubDomains'
[https @ 0x7efdb4001800] header='Date: Tue, 08 Jan 2019 14:53:47 GMT'
[https @ 0x7efdb4001800] header='Keep-Alive: timeout=60, max=4998'
[https @ 0x7efdb4001800] header='X-DM-LB-IP: 195.8.215.136'
[https @ 0x7efdb4001800] header='Expires: Tue, 08 Jan 2019 14:54:48 +0000'
[https @ 0x7efdb4001800] header='X-DM-LB-Name: lb-10'
[https @ 0x7efdb4001800] header='Content-Range: bytes 0-2775/2776'
[https @ 0x7efdb4001800] header='X-DM-BackNode-Response-Time: 32'
[https @ 0x7efdb4001800] header='Connection: close'
[https @ 0x7efdb4001800] header='Set-Cookie: dmvk=5c34b97c01f90; path=/; domain=.dailymotion.com'
[https @ 0x7efdb4001800] header='Set-Cookie: ts=415128; expires=Fri, 07 Feb 2020 14:53:47 GMT; Max-Age=34128000; path=/; domain=.dailymotion.com'
[https @ 0x7efdb4001800] header='Set-Cookie: v1st=91DEB00193F51C5CAF50579CA542B004; expires=Fri, 07 Feb 2020 14:53:47 GMT; max-age=34128000; path=/; domain=.dailymotion.com'
[https @ 0x7efdb4001800] header='X-Protected-By: Sqreen'
[https @ 0x7efdb4001800] header='X-Dm-Page: tn.html.cdn_director'
[https @ 0x7efdb4001800] header='Content-Length: 2776'
[https @ 0x7efdb4001800] header=''
Probing hls,applehttp score:100 size:2048
[hls,applehttp @ 0x7efdb4000b80] Format hls,applehttp probed with size=2048 and score=100
[hls,applehttp @ 0x7efdb4000b80] Opening 'https://proxy-010.ix7.dailymotion.com/sec(vKiHgyGzZBGev66P3WFPeIrlb_WdYJRrrxnWgJ36vhg)/video/414/178/420871414_mp4_h264_aac.m3u8#cell=core&qos_vpart=1&qos_geostore=1&qos_stail=1' for reading
[tcp @ 0x7efdb40a1540] Original list of addresses:sq=    0B f=0/0   
[tcp @ 0x7efdb40a1540] Address 188.65.126.10 port 443
[tcp @ 0x7efdb40a1540] Interleaved list of addresses:
[tcp @ 0x7efdb40a1540] Address 188.65.126.10 port 443
[tcp @ 0x7efdb40a1540] Starting connection attempt to 188.65.126.10 port 443
[tcp @ 0x7efdb40a1540] Successfully connected to 188.65.126.10 port 443
[https @ 0x7efdb4634a00] request: GET /sec(vKiHgyGzZBGev66P3WFPeIrlb_WdYJRrrxnWgJ36vhg)/video/414/178/420871414_mp4_h264_aac.m3u8#cell=core&qos_vpart=1&qos_geostore=1&qos_stail=1 HTTP/1.1
User-Agent: Lavf/58.20.100
Accept: */*
Connection: keep-alive
Host: proxy-010.ix7.dailymotion.com
Cookie: dmvk=5c34b97c01f90; ts=415128; v1st=91DEB00193F51C5CAF50579CA542B004
Icy-MetaData: 1


[https @ 0x7efdb4634a00] header='HTTP/1.1 403 Forbidden' 0B f=0/0   
[https @ 0x7efdb4634a00] http_code=403
[https @ 0x7efdb4634a00] HTTP error 403 Forbidden
[AVIOContext @ 0x7efdb4643100] Statistics: 2776 bytes read, 0 seeks
https://www.dailymotion.com/cdn/manifest/video/x6ykqgm.m3u8?sec=Hp1KzZeSUhvvwGdZmd4x5K8jkxgnOYC771Zyn8FZkJUN_AWj69qF7OeErQkRTW6D: Server returned 403 Forbidden (access denied)
    nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0

or:
ffplay $(youtube-dl -f http-720 -g https://www.dailymotion.com/video/x6ykqgm)
originaly reported at https://github.com/mpv-player/mpv/issues/6421.

Change History (7)

comment:1 by Carl Eugen Hoyos, 5 years ago

Component: undeterminedavformat
Keywords: http added

As promised, please confirm that this issue is reproducible with current FFmpeg git head.

comment:2 by remitamine, 5 years ago

./ffplay -v trace $(curl https://www.dailymotion.com/player/metadata/video/x6ykqgm | jq -r .qualities.auto[0].url)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11255    0 11255    0     0  18481      0 --:--:-- --:--:-- --:--:-- 18481
ffplay version N-92933-g2659a0fe36 Copyright (c) 2003-2019 the FFmpeg developers
  built with gcc 8.2.1 (GCC) 20181127
  configuration: --enable-gnutls
  libavutil      56. 25.100 / 56. 25.100
  libavcodec     58. 43.100 / 58. 43.100
  libavformat    58. 25.100 / 58. 25.100
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 46.101 /  7. 46.101
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
Initialized opengl renderer.
[NULL @ 0x7fbd04000b80] Opening 'https://www.dailymotion.com/cdn/manifest/video/x6ykqgm.m3u8?sec=Hp1KzZeSUhvvwGdZmd4x5EKntzC50U75CkThv1HpQa283G6re2xObR9BXhhiscBP' for reading
[https @ 0x7fbd04001780] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[tcp @ 0x7fbd04003d80] Original list of addresses:sq=    0B f=0/0   
[tcp @ 0x7fbd04003d80] Address 195.8.215.136 port 443
[tcp @ 0x7fbd04003d80] Interleaved list of addresses:
[tcp @ 0x7fbd04003d80] Address 195.8.215.136 port 443
[tcp @ 0x7fbd04003d80] Starting connection attempt to 195.8.215.136 port 443
[tcp @ 0x7fbd04003d80] Successfully connected to 195.8.215.136 port 443
[https @ 0x7fbd04001780] request: GET /cdn/manifest/video/x6ykqgm.m3u8?sec=Hp1KzZeSUhvvwGdZmd4x5EKntzC50U75CkThv1HpQa283G6re2xObR9BXhhiscBP HTTP/1.1
User-Agent: Lavf/58.25.100
Accept: */*
Range: bytes=0-
Connection: close
Host: www.dailymotion.com
Icy-MetaData: 1


[https @ 0x7fbd04001780] header='HTTP/1.1 206 Partial Content'0/0   
[https @ 0x7fbd04001780] http_code=206
[https @ 0x7fbd04001780] header='Content-Security-Policy-Report-Only: default-src https: blob: wss: data: android-webview-video-poster: android-webview: chrome-extension: safari-extension: 'unsafe-eval' 'unsafe-inline'; report-uri https://www.dailymotion.com/content_security_policy_report_uri'
[https @ 0x7fbd04001780] header='Server: DMS/1.0.42'
[https @ 0x7fbd04001780] header='X-DM-BackNode: web-609.adm.dc3.dailymotion.com:80'
[https @ 0x7fbd04001780] header='Vary: X-DM-SSL,Accept-Encoding'
[https @ 0x7fbd04001780] header='Cache-Control: private, max-age=60'
[https @ 0x7fbd04001780] header='Content-Type: application/vnd.apple.mpegurl'
[https @ 0x7fbd04001780] header='Content-Security-Policy: upgrade-insecure-requests'
[https @ 0x7fbd04001780] header='Access-Control-Expose-Headers: X-DM-BackNode-Response-Time'
[https @ 0x7fbd04001780] header='Strict-Transport-Security: max-age=31708800; includeSubDomains'
[https @ 0x7fbd04001780] header='Date: Tue, 08 Jan 2019 18:12:50 GMT'
[https @ 0x7fbd04001780] header='Keep-Alive: timeout=60, max=4989'
[https @ 0x7fbd04001780] header='X-DM-LB-IP: 195.8.215.136'
[https @ 0x7fbd04001780] header='Expires: Tue, 08 Jan 2019 18:13:50 +0000'
[https @ 0x7fbd04001780] header='X-DM-LB-Name: lb-07'
[https @ 0x7fbd04001780] header='Content-Range: bytes 0-2655/2656'
[https @ 0x7fbd04001780] header='X-DM-BackNode-Response-Time: 439'
[https @ 0x7fbd04001780] header='Connection: close'
[https @ 0x7fbd04001780] header='Set-Cookie: dmvk=5c34e8227efc1; path=/; domain=.dailymotion.com'
[https @ 0x7fbd04001780] header='Set-Cookie: ts=805906; expires=Fri, 07 Feb 2020 18:12:49 GMT; Max-Age=34128000; path=/; domain=.dailymotion.com'
[https @ 0x7fbd04001780] header='Set-Cookie: v1st=6AF047B2F90C9EA5257594119C01349B; expires=Fri, 07 Feb 2020 18:12:50 GMT; max-age=34128000; path=/; domain=.dailymotion.com'
[https @ 0x7fbd04001780] header='X-Protected-By: Sqreen'
[https @ 0x7fbd04001780] header='X-Dm-Page: tn.html.cdn_director'
[https @ 0x7fbd04001780] header='Content-Length: 2656'
[https @ 0x7fbd04001780] header=''
Probing hls,applehttp score:100 size:2048
[hls,applehttp @ 0x7fbd04000b80] Format hls,applehttp probed with size=2048 and score=100
[hls,applehttp @ 0x7fbd04000b80] Opening 'https://proxy-010.ix7.dailymotion.com/sec(vKiHgyGzZBGev66P3WFPeIrlb_WdYJRrrxnWgJ36vhg)/video/414/178/420871414_mp4_h264_aac.m3u8#cell=core&qos_vpart=1&qos_geostore=1' for reading
[tcp @ 0x7fbd04626a40] Original list of addresses:sq=    0B f=0/0   
[tcp @ 0x7fbd04626a40] Address 188.65.126.10 port 443
[tcp @ 0x7fbd04626a40] Interleaved list of addresses:
[tcp @ 0x7fbd04626a40] Address 188.65.126.10 port 443
[tcp @ 0x7fbd04626a40] Starting connection attempt to 188.65.126.10 port 443
[tcp @ 0x7fbd04626a40] Successfully connected to 188.65.126.10 port 443
[https @ 0x7fbd04629e80] request: GET /sec(vKiHgyGzZBGev66P3WFPeIrlb_WdYJRrrxnWgJ36vhg)/video/414/178/420871414_mp4_h264_aac.m3u8#cell=core&qos_vpart=1&qos_geostore=1 HTTP/1.1
User-Agent: Lavf/58.25.100
Accept: */*
Connection: keep-alive
Host: proxy-010.ix7.dailymotion.com
Cookie: dmvk=5c34e8227efc1; ts=805906; v1st=6AF047B2F90C9EA5257594119C01349B
Icy-MetaData: 1


[https @ 0x7fbd04629e80] header='HTTP/1.1 403 Forbidden' 0B f=0/0   
[https @ 0x7fbd04629e80] http_code=403
[https @ 0x7fbd04629e80] HTTP error 403 Forbidden
[AVIOContext @ 0x7fbd0462bcc0] Statistics: 2656 bytes read, 0 seeks
https://www.dailymotion.com/cdn/manifest/video/x6ykqgm.m3u8?sec=Hp1KzZeSUhvvwGdZmd4x5EKntzC50U75CkThv1HpQa283G6re2xObR9BXhhiscBP: Server returned 403 Forbidden (access denied)
    nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0

comment:4 by remitamine, 5 years ago

yes, the patch fixes the problem.

in reply to:  4 comment:5 by Steven Liu, 5 years ago

Replying to remitamine:

yes, the patch fixes the problem.

#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=460560,CODECS="mp4a.40.5,avc1.420016",RESOLUTION=512x288,NAME="380"
https://proxy-14.sv6.dailymotion.com/sec(vKiHgyGzZBGev66P3WFPeMcmv2VJ1r3V1wKJdUFBki0)/video/414/178/420871414_mp4_h264_aac.m3u8#cell=sv6&qos_vpart=1&qos_geostore=1
#EXT-X-STREAM-INF:BANDWIDTH=460560,CODECS="mp4a.40.5,avc1.420016",RESOLUTION=512x288,NAME="380"
https://proxy-33.sv6.dailymotion.com/sec(vKiHgyGzZBGev66P3WFPeMcmv2VJ1r3V1wKJdUFBki0)/video/414/178/420871414_mp4_h264_aac.m3u8#cell=sv6&qos_vpart=1&qos_geostore=1
#EXT-X-STREAM-INF:BANDWIDTH=105040,CODECS="mp4a.40.5,avc1.42000b",RESOLUTION=192x112,NAME="144"
https://proxy-14.sv6.dailymotion.com/sec(vy_cQlnqo1ExlDJIv6G22__IwYOD7PS7zZe4sY1KU0k)/video/414/178/420871414_mp4_h264_aac_l2.m3u8#cell=sv6&qos_vpart=1&qos_geostore=1
#EXT-X-STREAM-INF:BANDWIDTH=105040,CODECS="mp4a.40.5,avc1.42000b",RESOLUTION=192x112,NAME="144"
https://proxy-33.sv6.dailymotion.com/sec(vy_cQlnqo1ExlDJIv6G22__IwYOD7PS7zZe4sY1KU0k)/video/414/178/420871414_mp4_h264_aac_l2.m3u8#cell=sv6&qos_vpart=1&qos_geostore=1
#EXT-X-STREAM-INF:BANDWIDTH=246440,CODECS="mp4a.40.5,avc1.42000d",RESOLUTION=320x184,NAME="240"
https://proxy-14.sv6.dailymotion.com/sec(XnNqVw2L7-Wvn5PUa2qa5k_j-Uhw6f4Sz45r5nPcpAY)/video/414/178/420871414_mp4_h264_aac_ld.m3u8#cell=sv6&qos_vpart=1&qos_geostore=1
#EXT-X-STREAM-INF:BANDWIDTH=246440,CODECS="mp4a.40.5,avc1.42000d",RESOLUTION=320x184,NAME="240"
https://proxy-33.sv6.dailymotion.com/sec(XnNqVw2L7-Wvn5PUa2qa5k_j-Uhw6f4Sz45r5nPcpAY)/video/414/178/420871414_mp4_h264_aac_ld.m3u8#cell=sv6&qos_vpart=1&qos_geostore=1
#EXT-X-STREAM-INF:BANDWIDTH=836280,CODECS="mp4a.40.2,avc1.64001f",RESOLUTION=848x480,NAME="480"
https://proxy-14.sv6.dailymotion.com/sec(6kw0Rd7lKI8v-1_sTFBne3bPwxhuGLfEPlfqbguOZho)/video/414/178/420871414_mp4_h264_aac_hq.m3u8#cell=sv6&qos_vpart=1&qos_geostore=1
#EXT-X-STREAM-INF:BANDWIDTH=836280,CODECS="mp4a.40.2,avc1.64001f",RESOLUTION=848x480,NAME="480"
https://proxy-33.sv6.dailymotion.com/sec(6kw0Rd7lKI8v-1_sTFBne3bPwxhuGLfEPlfqbguOZho)/video/414/178/420871414_mp4_h264_aac_hq.m3u8#cell=sv6&qos_vpart=1&qos_geostore=1
#EXT-X-STREAM-INF:BANDWIDTH=2149280,CODECS="mp4a.40.2,avc1.64001f",RESOLUTION=1280x720,NAME="720"
https://proxy-14.sv6.dailymotion.com/sec(sc8SKNkr8tGmF7KYlSvnguPAMCWIQFWInc00XRlTA6I)/video/414/178/420871414_mp4_h264_aac_hd.m3u8#cell=sv6&qos_vpart=1&qos_geostore=1
#EXT-X-STREAM-INF:BANDWIDTH=2149280,CODECS="mp4a.40.2,avc1.64001f",RESOLUTION=1280x720,NAME="720"
https://proxy-33.sv6.dailymotion.com/sec(sc8SKNkr8tGmF7KYlSvnguPAMCWIQFWInc00XRlTA6I)/video/414/178/420871414_mp4_h264_aac_hd.m3u8#cell=sv6&qos_vpart=1&qos_geostore=1

What's the parameters cell=sv6&qos_vpart=1&qos_geostore=1 used?
QoS information?

comment:6 by Balling, 4 years ago

Status: newopen

comment:7 by Marton Balint, 4 years ago

Cc: Marton Balint added
Resolution: fixed
Status: openclosed

I just noticed this bug, this got fixed in 987ce96d41d21a120e6109f9bb1e3a9dcddbf30b.

Note: See TracTickets for help on using tickets.