Opened 11 years ago

Closed 11 years ago

Last modified 11 years ago

#2976 closed defect (fixed)

m3u8 playlist download fails if the lines in the manifests are longer than 1024 characters

Reported by: Jaime Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: hls
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

I'm using ffmpeg for watching Youtube's m3u8 streams and I've found that it fails if the lines in the manifest contains more than 1024 characters

How to reproduce:
using the version from git and commit 8ad2465987d31bb818235303c0b02b897cd02133, I use this script (requires youtube-dl):

URL=$(youtube-dl 'http://www.youtube.com/watch?v=rdcvn4W-6xM' -g)
echo $URL
./ffmpeg -y -i $URL -f mp4 hls.mp4

(You can manually find the manifest url in the video page)

The output I get is:

http://www.youtube.com/api/manifest/hls_playlist/id/add72f9f85beeb13/itag/96/source/youtube/ratebypass/yes/pfa/1/pbr/yes/hls_chunk_host/www.youtube.com/gir/yes/clen/5547517760/lmt/1347749241948586/dur/8870.400/pmbypass/yes/playlist_type/DVR/cp/U0hWTldTVl9NU0NONl9PTVpGOl9QaC1zZ1NuSnEx/maudio/1/sver/3/fexp/903802,927703,932204,928023,924606,929117,929121,929906,929907,929922,929923,929127,929129,929131,929930,936403,925724,925726,936310,925720,925722,925718,925714,929917,906945,929933,929935,939604,906842,927704,913428,919811,932309,913563,919373,930803,908536,938701,931924,934005,940501,936308,909549,901608,900816,912711,934507,912714,907231,936312,906001/upn/5htNr8XQdyc/ip/{my_ip}/ipbits/8/expire/1379788589/sparams/ip,ipbits,expire,id,itag,source,ratebypass,pfa,pbr,hls_chunk_host,gir,clen,lmt,dur,pmbypass,playlist_type,cp,maudio/signature/6EC01EDEEB7D161229D81AD728A68C9C1154ECAD.4D9721C03E7CC061A5093D83355957C08AACA696/key/dg_yt0/file/index.m3u8
ffmpeg version N-56538-g8ad2465 Copyright (c) 2000-2013 the FFmpeg developers
  built on Sep 21 2013 14:54:00 with llvm-gcc 4.2.1 (LLVM build 2336.11.00)
  configuration: --disable-ffplay --disable-ffprobe --disable-ffserver
  libavutil      52. 45.100 / 52. 45.100
  libavcodec     55. 33.100 / 55. 33.100
  libavformat    55. 18.102 / 55. 18.102
  libavdevice    55.  3.100 / 55.  3.100
  libavfilter     3. 86.102 /  3. 86.102
  libswscale      2.  5.100 /  2.  5.100
  libswresample   0. 17.103 /  0. 17.103
[http @ 0x7ff25a040600] HTTP error 403 Forbidden
Unable to open key file http://www.youtube.com/api/drm/hlskey?id=add72f9f85beeb13&itag=96&source=youtube&ratebypass=yes&pfa=1&pbr=yes&hls_chunk_host=www.youtube.com&gir=yes&clen=5547517760&lmt=1347749241948586&dur=8870.400&pmbypass=yes&playlist_type=DVR&cp=U0hWTldTVl9NU0NONl9PTVpGOl9QaC1zZ1NuSnEx&maudio=1&sver=3&fexp=903802,927703,932204,928023,924606,929117,929121,929906,929907,929922,929923,929127,929129,929131,929930,936403,925724,925726,936310,925720,925722,925718,925714,929917,906945,929933,929935,939604,906842,927704,913428,919811,932309,913563,919373,930803,908536,938701,931924,934005,940501,936308,909549,901608,900816,912711,934507,912714,907231,936312,906001&upn=5htNr8XQdyc&ip={my_ip}&ipbits=8&expire=1379788589&sparams=ip,ipbits,expire,id,itag,source,ratebypass,pfa,pbr,hls_chunk_host,gir,clen,lmt,dur,pmbypass,playlist_type,cp,maudio&signature=6EC01EDEEB7D161229D81AD728A68C9C1154ECAD.4D9721C03E7CC061A5093D83355957C08AACA696&key=dg_yt0&ek=00d1d2078d07e911274516317c3229cce139c648113fc4167a
[hls,applehttp @ 0x7ff25a03d800] Error when loading first segment 'http://www.youtube.com/videoplayback/id/add72f9f85beeb13/itag/96/source/youtube/range/0-1606463/file/seg.ts?ratebypass=yes&pfa=1&pbr=yes&hls_chunk_host=www.youtube.com&gir=yes&clen=5547517760&lmt=1347749241948586&dur=8870.400&pmbypass=yes&playlist_type=DVR&cp=U0hWTldTVl9NU0NONl9PTVpGOl9QaC1zZ1NuSnEx&maudio=1&sver=3&fexp=903802,927703,932204,928023,924606,929117,929121,929906,929907,929922,929923,929127,929129,929131,929930,936403,925724,925726,936310,925720,925722,925718,925714,929917,906945,929933,929935,939604,906842,927704,913428,919811,932309,913563,919373,930803,908536,938701,931924,934005,940501,936308,909549,901608,900816,912711,934507,912714,907231,936312,906001&upn=5htNr8XQdyc&ip={my_ip}&ipbits=8&expire=1379788589&sparams=ip,ipbits,expire,id,itag,source,ratebypass,pfa,pbr,hls_chunk_host,gir,clen,lmt,dur,pmbypass,playlist_type,cp,maudio&signature=6EC01EDEEB7D161229D81AD728A68C9C1154ECAD.4D9721C03E7CC061A5093D83355957C08AACA696&key=dg_yt0'
http://www.youtube.com/api/manifest/hls_playlist/id/add72f9f85beeb13/itag/96/source/youtube/ratebypass/yes/pfa/1/pbr/yes/hls_chunk_host/www.youtube.com/gir/yes/clen/5547517760/lmt/1347749241948586/dur/8870.400/pmbypass/yes/playlist_type/DVR/cp/U0hWTldTVl9NU0NONl9PTVpGOl9QaC1zZ1NuSnEx/maudio/1/sver/3/fexp/903802,927703,932204,928023,924606,929117,929121,929906,929907,929922,929923,929127,929129,929131,929930,936403,925724,925726,936310,925720,925722,925718,925714,929917,906945,929933,929935,939604,906842,927704,913428,919811,932309,913563,919373,930803,908536,938701,931924,934005,940501,936308,909549,901608,900816,912711,934507,912714,907231,936312,906001/upn/5htNr8XQdyc/ip/{my_ip}/ipbits/8/expire/1379788589/sparams/ip,ipbits,expire,id,itag,source,ratebypass,pfa,pbr,hls_chunk_host,gir,clen,lmt,dur,pmbypass,playlist_type,cp,maudio/signature/6EC01EDEEB7D161229D81AD728A68C9C1154ECAD.4D9721C03E7CC061A5093D83355957C08AACA696/key/dg_yt0/file/index.m3u8: Invalid data found when processing input

If I manually download the manifest and search for the key, it's longer:

http://www.youtube.com/api/drm/hlskey?id=add72f9f85beeb13&itag=96&source=youtube&ratebypass=yes&pfa=1&pbr=yes&hls_chunk_host=www.youtube.com&gir=yes&clen=5547517760&lmt=1347749241948586&dur=8870.400&pmbypass=yes&playlist_type=DVR&cp=U0hWTldTVl9NU0NONl9PTVpGOl9QaC1zZ1NuSnEx&maudio=1&sver=3&fexp=903802,927703,932204,928023,924606,929117,929121,929906,929907,929922,929923,929127,929129,929131,929930,936403,925724,925726,936310,925720,925722,925718,925714,929917,906945,929933,929935,939604,906842,927704,913428,919811,932309,913563,919373,930803,908536,938701,931924,934005,940501,936308,909549,901608,900816,912711,934507,912714,907231,936312,906001&upn=5htNr8XQdyc&ip={my_ip}&ipbits=8&expire=1379788589&sparams=ip,ipbits,expire,id,itag,source,ratebypass,pfa,pbr,hls_chunk_host,gir,clen,lmt,dur,pmbypass,playlist_type,cp,maudio&signature=6EC01EDEEB7D161229D81AD728A68C9C1154ECAD.4D9721C03E7CC061A5093D83355957C08AACA696&key=dg_yt0&ek=00d1d2078d07e911274516317c3229cce139c648113fc4167a59e8a64d87b0a8f11877de7e0197d2f1a49d1639eed28a5d1785639ac94664b328641b824f55413a88c43da5b88525b0

After looking into libavformat/hls.c I think I've found the problem, in the line 215

    char line[1024];

This causes lines to be trunctated, and therefore the key url is wrong. If I change the value to something higher, like 1500, the key url is correctly extracted and everythin works.

Change History (3)

comment:1 by Carl Eugen Hoyos, 11 years ago

Component: undeterminedavformat
Keywords: hls added
Status: newopen

comment:2 by Michael Niedermayer, 11 years ago

Reproduced by developer: set
Resolution: fixed
Status: openclosed

comment:3 by Jaime, 11 years ago

Thanks! it works now.

Note: See TracTickets for help on using tickets.