Opened 2 years ago

Closed 2 years ago

#5287 closed defect (fixed)

Protocol not on whitelist 'file'! in m3u8 for local use

Reported by: remitamine Owned by:
Priority: important Component: avformat
Version: git-master Keywords: hls regression
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

i know that file protocol has been disabled for security conserns but for local use this is not an issue.
i have a script in python that parse m3u8 manifests and make a version that refrance local segments path and than start downloading segments in parallel(using aria2c batch option) for speeding the download and in the same time to be able to resume the download at later time and i can start watch directly while segments are downloading.
it was working before untill the version 3 released witch disable the file protocol in m3u8 by default, but it's not problem for the case that i've described.

Attachments (1)

The Colbeard.tar.bz (2.0 MB) - added by cehoyos 2 years ago.

Download all attachments as: .zip

Change History (16)

comment:1 Changed 2 years ago by cehoyos

How can I reproduce your issue?

comment:2 Changed 2 years ago by remitamine

this is the link of the script:
https://gist.github.com/remitamine/2e0cbef7d50e8e7ef2d2
it depends on youtube-dl.
try to download any m3u8 url or any site supported by youtube-dl that has only m3u8 formats (cbs, nbc, history, dcn...)
it will download the segments and the create an m3u8 file that link to the local segements.
something like this:

#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-ALLOW-CACHE:YES
#EXT-X-KEY:METHOD=AES-128,URI="Connect Chat feat. Garth Brooks/crypt.key"
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment1_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment2_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment3_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment4_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment5_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment6_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment7_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment8_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment9_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment10_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment11_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment12_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment13_1_av.ts

i was able to play the file with versions before 3.0, but now it didn't work.

[crypto @ 0x7fc00003b120] Protocol not on whitelist 'file'! f=0/0   
[hls,applehttp @ 0x7fc000009240] Failed to open segment of playlist 0
[crypto @ 0x7fc00003b2c0] Protocol not on whitelist 'file'!
[hls,applehttp @ 0x7fc000009240] Failed to open segment of playlist 0
[crypto @ 0x7fc00003b3c0] Protocol not on whitelist 'file'!

comment:3 Changed 2 years ago by gjdfgh

This demuxer is for HLS, not anything else.

I suggest closing as invalid.

Get a real player (I heard vlc is pretty popular).

comment:4 Changed 2 years ago by remitamine

it's the same for vlc.

[00007f00d4c01828] httplive stream error: segment 0 should have been available (stream 0)
[00007f00d4c01828] httplive stream error: segment 0 should have been available (stream 0)
[00007f00d4c01828] httplive stream error: segment 0 should have been available (stream 0)
[00007f00d4c01828] httplive stream error: segment 0 should have been available (stream 0)
[00007f00d4c01828] httplive stream error: segment 0 should have been available (stream 0)
[00007f00d4c01828] httplive stream error: segment 0 should have been available (stream 0)

it can't play the result file and also mpv

Last edited 2 years ago by remitamine (previous) (diff)

comment:5 Changed 2 years ago by gjdfgh

Because they interpret it as HLS, not as playlist.

So stop creating invalid HLS files, and use .m3u as extension or something.

Edit: file extension alone is probably not enough. You also need to remove the EXT-X headers.

Last edited 2 years ago by gjdfgh (previous) (diff)

comment:6 Changed 2 years ago by gjdfgh

Also, looking a bit closer, maybe I misunderstood what the youtube-dl thing does. Does it just download HLS from remote, without changing the HLS file structure and metadata itself? (But then why use youtube-dl at all?)

In that case, it's actually a somewhat valid use-case: play valid HLS from a filesystem.

comment:7 Changed 2 years ago by cehoyos

I still don't understand how I can reproduce this issue: You pointed to a script (which is generally bad) but what am I supposed to do with the script to reproduce the issue you see?

comment:8 Changed 2 years ago by remitamine

cehoyos, if you download an m3u8 manifest using the script it will produce a m3u8 file simular to the example above.
for example you can try this:

python3 m3u8_downloader.py http://colbertlateshow.com/video/8GmB0oY0McANFvp2aEffk9jZZZ2YyXxy/the-colbeard/

if you try to convert it (The Colbeard.m3u8) using ffmpeg 3 it won't work but it works with previous versions.
i ask at at least to enable file protocol if the input playlist is also requested using a local file but disable it if it's requested from an http request.

gjdfgh, i use youtube-dl to extract the the m3u8 url from diffrent sites, i know that i can download them directly using youtube-dl but currently it uses ffmpeg to download HLS protected manifests and ffmpeg can't resume the download and it's also slow(it uses one connection).
in the script i can resume the download of HLS protected manifests and speed up the download(parrallel downloads if i change the -j param passed to aria2c) and in the same time i can play the video.

Last edited 2 years ago by remitamine (previous) (diff)

comment:9 Changed 2 years ago by remitamine

gjdfgh, about using .m3u ffmpeg won't be able to use the file if i remove EXT-X-KEY which tell ffmpeg how to decrypt the segments.

comment:10 Changed 2 years ago by cehoyos

Is the script necessary to reproduce the issue?
Or can you attach an m3u8 file and a transport stream (cut with dd) that allow to reproduce the problem?

comment:11 follow-up: Changed 2 years ago by remitamine

no it's not necessary it just simplify to generate the m3u8 file and download the segments
i will try to compress an m3u8 file with some segments and try to upload them.

Last edited 2 years ago by cehoyos (previous) (diff)

comment:12 in reply to: ↑ 11 Changed 2 years ago by cehoyos

Replying to remitamine:

i will try to compress an m3u8 file with some segments and try to upload them.

Please use dd to cut one segment.

comment:13 Changed 2 years ago by remitamine

i already uploaded a file with one segment.
http://www.datafilehost.com/d/51aadaa8

comment:14 Changed 2 years ago by cehoyos

  • Component changed from undetermined to avformat
  • Keywords regression added
  • Priority changed from normal to important
  • Reproduced by developer set
  • Status changed from new to open
  • Version changed from unspecified to git-master

For future tickets: Please always provide a command line that allows to reproduce your issue together with the complete, uncut console output.

Regression since fe3fed0b143ef6bf2d9b65ce05d55aba4224429e
The error message has changed since.

$ ffmpeg -i The\ Colbeard.m3u8 -qscale 2 out.avi
ffmpeg version N-78961-gda904fa Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      55. 19.100 / 55. 19.100
  libavcodec     57. 28.100 / 57. 28.100
  libavformat    57. 28.100 / 57. 28.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 39.102 /  6. 39.102
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
[hls,applehttp @ 0x30a53c0] Failed to open segment of playlist 0
    Last message repeated 34 times
[hls,applehttp @ 0x30a53c0] Error when loading first segment 'The Colbeard/segment1_1_av.ts'
The Colbeard.m3u8: Invalid data found when processing input

Changed 2 years ago by cehoyos

comment:15 Changed 2 years ago by michael

  • Resolution set to fixed
  • Status changed from open to closed
Note: See TracTickets for help on using tickets.