Opened 6 years ago

Last modified 2 weeks ago

#5375 open enhancement

Can't download a m3u playlist stream that uses AES encryption

Reported by: viv1d-shadows Owned by:
Priority: wish Component: avformat
Version: git-master Keywords: hls
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug: I am trying to download a video using a m3u8 playlist but I get the error saying "Sample-AES encryption is not yet supported; Failed to open segment of playlist 0". Then at the end I get "Error loading first segment: <very long url is here> Invalid data when processing input
How to reproduce:

% ffmpeg -i "https://once.unicornmedia.com/now/od/aut
o/e62f13e1-cab4-4fca-bdbb-eaa84d793f41/523587ee-47d2-403b-a3d0-e5db30f6929c/cim
44emqp001i0fo5daa1yd3o/content.m3u8" -c copy test.ts

ffmpeg version: 3.0
built on: Unknown

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

Change History (17)

comment:1 by Carl Eugen Hoyos, 6 years ago

Keywords: hls added

Please test current FFmpeg git head and provide the failing command line together with the complete, uncut console output to make this a valid ticket.

comment:2 by viv1d-shadows, 6 years ago

$ ffmpeg -i "https://once.unicornmedia.com/now/od/auto/e62f13e1-cab4-4fca-bdbb-eaa84d793f41/523587ee-47d2-403b-a3d0-e5db30f6929c/cim44emqp001i0fo5daa1yd3o/content.m3u8" -c copy test.ts
ffmpeg version 3.0 Copyright (c) 2000-2016 the FFmpeg developers
  built with Apple LLVM version 7.0.2 (clang-700.1.81)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.0 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-opencl --enable-libx264 --enable-libmp3lame --enable-libxvid --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libspeex --enable-openssl --enable-libvidstab --enable-libx265 --enable-nonfree --enable-vda
  libavutil      55. 17.103 / 55. 17.103
  libavcodec     57. 24.102 / 57. 24.102
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 31.100 /  6. 31.100
  libavresample   3.  0.  0 /  3.  0.  0
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
[hls,applehttp @ 0x7fd4da810600] SAMPLE-AES encryption is not supported yet
[hls,applehttp @ 0x7fd4da810600] Failed to open segment of playlist 0
[hls,applehttp @ 0x7fd4da810600] SAMPLE-AES encryption is not supported yet
[hls,applehttp @ 0x7fd4da810600] Failed to open segment of playlist 0

...

[hls,applehttp @ 0x7fd4da810600] SAMPLE-AES encryption is not supported yet
[hls,applehttp @ 0x7fd4da810600] Failed to open segment of playlist 0
[hls,applehttp @ 0x7fd4da810600] SAMPLE-AES encryption is not supported yet
[hls,applehttp @ 0x7fd4da810600] Failed to open segment of playlist 0
[hls,applehttp @ 0x7fd4da810600] Error when loading first segment 'https://api002-phx.unicornmedia.com/now/media/segment/e62f13e1-cab4-4fca-bdbb-eaa84d793f41/523587ee-47d2-403b-a3d0-e5db30f6929c/74ba4d0b-a347-11e4-bfdb-005056837bc7/d4ddcab5-a383-4e99-af17-9e16c7be623b/0/0/270/content.ts?visitguid=6442f8e8-b6d5-4c77-a7b8-9e82789b1ce7&baseguid=d4ddcab5-a383-4e99-af17-9e16c7be623b&streamDuration=3297&umx=cAw7b/V1pNitZ2XV9RnbyA==&startsegmentseconds=0&endsegmentseconds=10'
https://once.unicornmedia.com/now/od/auto/e62f13e1-cab4-4fca-bdbb-eaa84d793f41/523587ee-47d2-403b-a3d0-e5db30f6929c/cim44emqp001i0fo5daa1yd3o/content.m3u8: Invalid data found when processing input
Last edited 6 years ago by Carl Eugen Hoyos (previous) (diff)

comment:3 by Carl Eugen Hoyos, 6 years ago

Component: undeterminedavformat
Priority: normalwish
Status: newopen
Type: defectenhancement
Version: unspecifiedgit-master

For future tickets: Please remember to test current FFmpeg git head before reporting issues.

comment:4 by viv1d-shadows, 6 years ago

I did try to use the current git snapshot, but after I compiled it on my ubuntu based machine I get the error saying "error while loading shared libraries: libx264.so.148" hence why I didn't use the git snapshot because I don't know how to get rid of that error.

Do I need to do anything else to this ticket, other than trying to test the git build?

Last edited 6 years ago by viv1d-shadows (previous) (diff)

in reply to:  4 comment:5 by Carl Eugen Hoyos, 6 years ago

Replying to viv1d-shadows:

I did try to use the current git snapshot, but after I compiled it on my ubuntu based machine I get the error saying "error while loading shared libraries: libx264.so.148"

This error can never occur for a default compilation which should allow to reproduce this issue.

comment:6 by viv1d-shadows, 6 years ago

Ok I will try it again later today and get back to you

in reply to:  6 comment:7 by Carl Eugen Hoyos, 6 years ago

Replying to viv1d-shadows:

Ok I will try it again later today and get back to you

When I wrote For future tickets I meant that you should follow my advice when reporting your next issue.

comment:8 by viv1d-shadows, 6 years ago

Yeah I understand. So I am required to do anything else to this ticket, or just wait until the developers come around and look at this issue?

Cheers

comment:9 by Carl Eugen Hoyos, 6 years ago

Ideally you would fix the issue and send a patch to the development mailing list.

in reply to:  8 comment:10 by James, 6 years ago

Replying to viv1d-shadows:

Yeah I understand. So I am required to do anything else to this ticket, or just wait until the developers come around and look at this issue?

No, the ticket is fine as is. But keep in mind this is a feature request. It being implemented depends on someone wanting to give this a go.

comment:11 by fitz123, 5 years ago

+1 for this feature.
Version 3.1.3 and it's still not here =(

comment:12 by marantz, 5 years ago

sub

comment:13 by kvasa, 4 years ago

Hi, I'm no expert, but isn't this pretty much what is needed? https://github.com/selsta/hlsdl/blob/master/src/hls.c#L292

It is under MIT license and seems to be doing SAMPLE-AES decryption using libavformat and libavcodec. Unfortunately, I do not have the skills to implement this directly into FFmpeg. :(

comment:14 by CoRoNe, 4 years ago

Today the NPO (Dutch Public Broadcast) has started using the SAMPLE-AES encryption which FFmpeg sadly enough still doesn't support:

D:\Binaries>FOR /F "delims=" %A IN ('xidel.exe -s "https://www.npo.nl/nos-journaal/27-10-2017/POW_03375731" -e "let $prid:=extract($url,'.+/(.+)',1) return json(concat('https://start-player.npo.nl/video/',$prid,'/streams?profile=hls&quality=npo&tokenId=',x:request({'data':'https://www.npo.nl/player/'||$prid,'method':'POST'})/json/token))//src"') DO ffmpeg-N-88268-g75bd010_hls-test.exe -v 40 -i "%A"

D:\Binaries>ffmpeg-N-88268-g75bd010_hls-test.exe -v 40 -i "https://nl-ams-p4-am5.cdn.streamgate.nl/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE1MDkzNDU3NjMsImNsaWVudF9pcCI6IjgwLjEwMS40OS41IiwidXJpIjoiXC92b2RcL25wb1wvZnBzXC9URVNUXC9ucG9cL2hsc1wvUE9XXzAzMzc1NzMxIn0.8TcJZK13155360BpwACs3YJepA-TAFkoD4RjcdsSXT8/vod/npo/fps/TEST/npo/hls/POW_03375731/playlist.m3u8"
ffmpeg version N-88268-g75bd010 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --arch=x86 --target-os=mingw32 --cross-prefix=/cygdrive/m/ffmpeg-windows-build-helpers-master/native_build/windows/ffmpeg_local_builds/sandbox/cross_compilers/mingw-w64-i686/bin/i686-w64-mingw32- --pkg-config=pkg-config --pkg-config-flags=--static --enable-gray --enable-version3 --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-schannel --disable-txtpages --disable-w32threads --disable-ffprobe --disable-hwaccels --disable-muxers --disable-bsfs --disable-filters --enable-gnutls --extra-libs=-lcrypt32 --enable-gpl --extra-cflags='-march=pentium3' --extra-cflags=-O2 --extra-cflags='-mfpmath=sse' --extra-cflags=-msse --enable-static --disable-shared --prefix=/cygdrive/m/ffmpeg-windows-build-helpers-master/native_build/windows/ffmpeg_local_builds/sandbox/cross_compilers/mingw-w64-i686/i686-w64-mingw32
  libavutil      56.  0.100 / 56.  0.100
  libavcodec     58.  1.100 / 58.  1.100
  libavformat    58.  0.102 / 58.  0.102
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  0.101 /  7.  0.101
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.100 /  3.  0.100
  libpostproc    55.  0.100 / 55.  0.100
[hls,applehttp @ 02539440] Opening 'https://nl-ams-p4-am5.cdn.streamgate.nl/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE1MDkzNDU3NjMsImNsaWVudF9pcCI6IjgwLjEwMS40OS41IiwidXJpIjoiXC92b2RcL25wb1wvZnBzXC9URVNUXC9ucG9cL2hsc1wvUE9XXzAzMzc1NzMxIn0.8TcJZK13155360BpwACs3YJepA-TAFkoD4RjcdsSXT8/vod/npo/fps/TEST/npo/hls/POW_03375731/a1/a1.m3u8' for reading
[hls,applehttp @ 02539440] Opening 'https://nl-ams-p4-am5.cdn.streamgate.nl/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE1MDkzNDU3NjMsImNsaWVudF9pcCI6IjgwLjEwMS40OS41IiwidXJpIjoiXC92b2RcL25wb1wvZnBzXC9URVNUXC9ucG9cL2hsc1wvUE9XXzAzMzc1NzMxIn0.8TcJZK13155360BpwACs3YJepA-TAFkoD4RjcdsSXT8/vod/npo/fps/TEST/npo/hls/POW_03375731/v2/v2.m3u8' for reading
[hls,applehttp @ 02539440] Opening 'https://nl-ams-p4-am5.cdn.streamgate.nl/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE1MDkzNDU3NjMsImNsaWVudF9pcCI6IjgwLjEwMS40OS41IiwidXJpIjoiXC92b2RcL25wb1wvZnBzXC9URVNUXC9ucG9cL2hsc1wvUE9XXzAzMzc1NzMxIn0.8TcJZK13155360BpwACs3YJepA-TAFkoD4RjcdsSXT8/vod/npo/fps/TEST/npo/hls/POW_03375731/v3/v3.m3u8' for reading
[hls,applehttp @ 02539440] Opening 'https://nl-ams-p4-am5.cdn.streamgate.nl/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE1MDkzNDU3NjMsImNsaWVudF9pcCI6IjgwLjEwMS40OS41IiwidXJpIjoiXC92b2RcL25wb1wvZnBzXC9URVNUXC9ucG9cL2hsc1wvUE9XXzAzMzc1NzMxIn0.8TcJZK13155360BpwACs3YJepA-TAFkoD4RjcdsSXT8/vod/npo/fps/TEST/npo/hls/POW_03375731/v4/v4.m3u8' for reading
[hls,applehttp @ 02539440] Opening 'https://nl-ams-p4-am5.cdn.streamgate.nl/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE1MDkzNDU3NjMsImNsaWVudF9pcCI6IjgwLjEwMS40OS41IiwidXJpIjoiXC92b2RcL25wb1wvZnBzXC9URVNUXC9ucG9cL2hsc1wvUE9XXzAzMzc1NzMxIn0.8TcJZK13155360BpwACs3YJepA-TAFkoD4RjcdsSXT8/vod/npo/fps/TEST/npo/hls/POW_03375731/v5/v5.m3u8' for reading
[hls,applehttp @ 02539440] HLS request for url 'https://nl-ams-p4-am5.cdn.streamgate.nl/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE1MDkzNDU3NjMsImNsaWVudF9pcCI6IjgwLjEwMS40OS41IiwidXJpIjoiXC92b2RcL25wb1wvZnBzXC9URVNUXC9ucG9cL2hsc1wvUE9XXzAzMzc1NzMxIn0.8TcJZK13155360BpwACs3YJepA-TAFkoD4RjcdsSXT8/vod/npo/fps/TEST/npo/hls/POW_03375731/a1/fileSequence.aac', offset 0, playlist 0
[hls,applehttp @ 02539440] SAMPLE-AES encryption is not supported yet
[hls,applehttp @ 02539440] Failed to open segment of playlist 0
[hls,applehttp @ 02539440] HLS request for url 'https://nl-ams-p4-am5.cdn.streamgate.nl/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE1MDkzNDU3NjMsImNsaWVudF9pcCI6IjgwLjEwMS40OS41IiwidXJpIjoiXC92b2RcL25wb1wvZnBzXC9URVNUXC9ucG9cL2hsc1wvUE9XXzAzMzc1NzMxIn0.8TcJZK13155360BpwACs3YJepA-TAFkoD4RjcdsSXT8/vod/npo/fps/TEST/npo/hls/POW_03375731/a1/fileSequence.aac', offset 131089, playlist 0
[hls,applehttp @ 02539440] SAMPLE-AES encryption is not supported yet
[hls,applehttp @ 02539440] Failed to open segment of playlist 0
[...]
[hls,applehttp @ 02539440] HLS request for url 'https://nl-ams-p4-am5.cdn.streamgate.nl/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE1MDkzNDU3NjMsImNsaWVudF9pcCI6IjgwLjEwMS40OS41IiwidXJpIjoiXC92b2RcL25wb1wvZnBzXC9URVNUXC9ucG9cL2hsc1wvUE9XXzAzMzc1NzMxIn0.8TcJZK13155360BpwACs3YJepA-TAFkoD4RjcdsSXT8/vod/npo/fps/TEST/npo/hls/POW_03375731/a1/fileSequence.aac', offset 24451598, playlist 0
[hls,applehttp @ 02539440] SAMPLE-AES encryption is not supported yet
[hls,applehttp @ 02539440] Failed to open segment of playlist 0
Format aac detected only with low score of 1, misdetection possible!
[hls,applehttp @ 02539440] No longer receiving playlist 0
https://nl-ams-p4-am5.cdn.streamgate.nl/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE1MDkzNDU3NjMsImNsaWVudF9pcCI6IjgwLjEwMS40OS41IiwidXJpIjoiXC92b2RcL25wb1wvZnBzXC9URVNUXC9ucG9cL2hsc1wvUE9XXzAzMzc1NzMxIn0.8TcJZK13155360BpwACs3YJepA-TAFkoD4RjcdsSXT8/vod/npo/fps/TEST/npo/hls/POW_03375731/playlist.m3u8: Invalid data found when processing input

Compiling this hlsdl didn't go so well either.

comment:15 by intermediary, 2 years ago

A current git compile of ffmpeg still has trouble handling streams with sample-aes.
HLSDL is also a bit troublesome with getting the right key material in.
Hopefully someone will come around and implement this in ffmpeg proper.

comment:17 by Balling, 2 weeks ago

The patches are applied now. Please test. ff958b38463f6ebd17471c40001287c71698f639

-c copy does work with https://pw.pc.cdn.bitgravity.com/b7826d86-612d-4ef2-b17c-cc4f91d9584c/master.m3u8 but you still need a key, alas.

-allowed_extensions ALL does not help either.

Last edited 2 weeks ago by Balling (previous) (diff)
Note: See TracTickets for help on using tickets.