Protocol not on whitelist 'file'! in m3u8 for local use

[remitamine]

i know that file protocol has been disabled for security conserns but for local use this is not an issue.
i have a script in python that parse m3u8 manifests and make a version that refrance local segments path and than start downloading segments in parallel(using aria2c batch option) for speeding the download and in the same time to be able to resume the download at later time and i can start watch directly while segments are downloading.
it was working before untill the version 3 released witch disable the file protocol in m3u8 by default, but it's not problem for the case that i've described.

[Carl Eugen Hoyos]

How can I reproduce your issue?

[remitamine]

this is the link of the script:
​https://gist.github.com/remitamine/2e0cbef7d50e8e7ef2d2
it depends on youtube-dl.
try to download any m3u8 url or any site supported by youtube-dl that has only m3u8 formats (cbs, nbc, history, dcn...)
it will download the segments and the create an m3u8 file that link to the local segements.
something like this:

#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-ALLOW-CACHE:YES
#EXT-X-KEY:METHOD=AES-128,URI="Connect Chat feat. Garth Brooks/crypt.key"
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment1_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment2_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment3_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment4_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment5_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment6_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment7_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment8_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment9_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment10_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment11_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment12_1_av.ts
#EXTINF:10.000,
Connect Chat feat. Garth Brooks/segment13_1_av.ts

i was able to play the file with versions before 3.0, but now it didn't work.

[crypto @ 0x7fc00003b120] Protocol not on whitelist 'file'! f=0/0   
[hls,applehttp @ 0x7fc000009240] Failed to open segment of playlist 0
[crypto @ 0x7fc00003b2c0] Protocol not on whitelist 'file'!
[hls,applehttp @ 0x7fc000009240] Failed to open segment of playlist 0
[crypto @ 0x7fc00003b3c0] Protocol not on whitelist 'file'!

[gjdfgh]

This demuxer is for HLS, not anything else.

I suggest closing as invalid.

Get a real player (I heard vlc is pretty popular).

[remitamine]

it's the same for vlc.

[00007f00d4c01828] httplive stream error: segment 0 should have been available (stream 0)
[00007f00d4c01828] httplive stream error: segment 0 should have been available (stream 0)
[00007f00d4c01828] httplive stream error: segment 0 should have been available (stream 0)
[00007f00d4c01828] httplive stream error: segment 0 should have been available (stream 0)
[00007f00d4c01828] httplive stream error: segment 0 should have been available (stream 0)
[00007f00d4c01828] httplive stream error: segment 0 should have been available (stream 0)

[gjdfgh]

Because they interpret it as HLS, not as playlist.

So stop creating invalid HLS files, and use .m3u as extension or something.

Edit: file extension alone is probably not enough. You also need to remove the EXT-X headers.

[gjdfgh]

Also, looking a bit closer, maybe I misunderstood what the youtube-dl thing does. Does it just download HLS from remote, without changing the HLS file structure and metadata itself? (But then why use youtube-dl at all?)

In that case, it's actually a somewhat valid use-case: play valid HLS from a filesystem.

[Carl Eugen Hoyos]

I still don't understand how I can reproduce this issue: You pointed to a script (which is generally bad) but what am I supposed to do with the script to reproduce the issue you see?

[remitamine]

cehoyos, if you download an m3u8 manifest using the script it will produce a m3u8 file simular to the example above.
for example you can try this:

python3 m3u8_downloader.py http://colbertlateshow.com/video/8GmB0oY0McANFvp2aEffk9jZZZ2YyXxy/the-colbeard/

if you try to convert it (The Colbeard.m3u8) using ffmpeg 3 it won't work but it works with previous versions.
i ask at at least to enable file protocol if the input playlist is also requested using a local file but disable it if it's requested from an http request.

gjdfgh, i use youtube-dl to extract the the m3u8 url from diffrent sites, i know that i can download them directly using youtube-dl but currently it uses ffmpeg to download HLS protected manifests and ffmpeg can't resume the download and it's also slow(it uses one connection).
in the script i can resume the download of HLS protected manifests and speed up the download(parrallel downloads if i change the -j param passed to aria2c) and in the same time i can play the video.

[remitamine]

gjdfgh, about using .m3u ffmpeg won't be able to use the file if i remove EXT-X-KEY which tell ffmpeg how to decrypt the segments.

[Carl Eugen Hoyos]

Is the script necessary to reproduce the issue?
Or can you attach an m3u8 file and a transport stream (cut with dd) that allow to reproduce the problem?

[remitamine]

no it's not necessary it just simplify to generate the m3u8 file and download the segments
i will try to compress an m3u8 file with some segments and try to upload them.

[Carl Eugen Hoyos]

Replying to remitamine:

i will try to compress an m3u8 file with some segments and try to upload them.

Please use dd to cut one segment.

[remitamine]

i already uploaded a file with one segment.
​http://www.datafilehost.com/d/51aadaa8

[Carl Eugen Hoyos]

For future tickets: Please always provide a command line that allows to reproduce your issue together with the complete, uncut console output.

Regression since fe3fed0b143ef6bf2d9b65ce05d55aba4224429e
The error message has changed since.

$ ffmpeg -i The\ Colbeard.m3u8 -qscale 2 out.avi
ffmpeg version N-78961-gda904fa Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      55. 19.100 / 55. 19.100
  libavcodec     57. 28.100 / 57. 28.100
  libavformat    57. 28.100 / 57. 28.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 39.102 /  6. 39.102
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
[hls,applehttp @ 0x30a53c0] Failed to open segment of playlist 0
    Last message repeated 34 times
[hls,applehttp @ 0x30a53c0] Error when loading first segment 'The Colbeard/segment1_1_av.ts'
The Colbeard.m3u8: Invalid data found when processing input

[Michael Niedermayer]

Fixed in efa98cdc2ff17a2f3b0ceb69e22864fd5bc433db