Request support for decoding / demuxing Adobe HDS dynamic http streaming

[Otto D]

Adobe HDS is an adaptive streaming format used primarily to deliver video streams through content delivery networks.

It uses a manifest file (F4F) to describe the segments of a file, and then it adaptively delivers segments and sequences in the "optimal" bitrate depending on the client's bandwith and the total server load.

Certain major content providers are moving to this format (at least in Sweden) and it would be really great if ffmpeg could support it.

A good summary of HDS is found at:
​http://rdkls.blogspot.se/2011/11/what-i-know-about-http-adaptive.html

It seems that most projects that do decode such streams use a PHP script (!) found at:
​https://github.com/K-S-V/Scripts/blob/master/AdobeHDS.php

The Open Source Media Player project has action script code which probably does the same thing:
​http://sourceforge.net/adobe/osmf/

The format is quite similar to applehttp/hls, so it should be possible to borrow some patterns from the support for that format, which is already in libavformat.

I am a C# / java developer and could probably make this happen in C, but it would take a lot of effort. I am not yet familiar with the ffmpeg source code. So I'm hoping that someone might already be working on this? Or that someone well versed on avformat development can take this on. I'd be happy to contribute with my own efforts.

Best regards!

[Carl Eugen Hoyos]

Does FFmpeg support playing the segments / the first transmitted segment?

[Otto D]

No.

Here is the output from when I attempt to play a stream, using the recent static build for 64 bit windows:

>ffmpeg.exe -i "http://svtplay6s-f.akamaihd.net/z/se
/secure/20121118/1232796-008A/EARTHLY_TREASUR-008A-7a4e5968ffc1c170_,900,320,420
,620,1660,2760,.mp4.csmil/manifest.f4m?hdcore=2.8.0&g=LLIOFRVAXRWA" D:\tmp\fmt-t
est.ts
ffmpeg version N-46469-gc995644 Copyright (c) 2000-2012 the FFmpeg developers
  built on Nov  5 2012 17:58:15 with gcc 4.7.2 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-pthreads --enable-runt
ime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libass -
-enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enab
le-libgsm --enable-libmp3lame --enable-libnut --enable-libopenjpeg --enable-libo
pus --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheo
ra --enable-libutvideo --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-li
bvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --ena
ble-zlib
  libavutil      52.  5.100 / 52.  5.100
  libavcodec     54. 71.100 / 54. 71.100
  libavformat    54. 36.100 / 54. 36.100
  libavdevice    54.  3.100 / 54.  3.100
  libavfilter     3. 21.106 /  3. 21.106
  libswscale      2.  1.102 /  2.  1.102
  libswresample   0. 16.100 /  0. 16.100
  libpostproc    52.  1.100 / 52.  1.100
http://svtplay6s-f.akamaihd.net/z/se/secure/20121118/1232796-008A/EARTHLY_TREASU
R-008A-7a4e5968ffc1c170_,900,320,420,620,1660,2760,.mp4.csmil/manifest.f4m?hdcor
e=2.8.0&g=LLIOFRVAXRWA: Invalid data found when processing input

If I inspect the contents of that URL, I get the F4M manifest:

<?xml version="1.0" encoding="UTF-8"?>
<manifest xmlns="http://ns.adobe.com/f4m/1.0" xmlns:akamai="uri:akamai.com/f4m/1.0">
  <akamai:version>2.0</akamai:version>
  <akamai:bw>5000</akamai:bw>
  <id>/se/secure/20121118/1232796-008A/EARTHLY_TREASUR-008A-7a4e5968ffc1c170_,900,320,420,620,1660,2760,.mp4.csmil_0</id>
  <streamType>recorded</streamType>
  <akamai:streamType>vod</akamai:streamType>
  <duration>300.501</duration>
  <streamBaseTime>0.000</streamBaseTime>
  <bootstrapInfo profile="named" id="bootstrap_1">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEldUAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGWUAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <bootstrapInfo profile="named" id="bootstrap_2">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEldUAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGWUAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <bootstrapInfo profile="named" id="bootstrap_3">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEldUAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGWUAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <bootstrapInfo profile="named" id="bootstrap_0">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEldUAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGWUAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <bootstrapInfo profile="named" id="bootstrap_4">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEldUAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGWUAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <bootstrapInfo profile="named" id="bootstrap_5">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAElcAAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGVAAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <media bitrate="315" url="1_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_1">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcsgEGJN0vAAFd2lkdGgAQHQAAAAAAAAABmhlaWdodABAZoAAAAAAAAANdmlkZW9kYXRhcmF0ZQBAY/imU7h9mQAJZnJhbWVyYXRlAEA4/42A0ljSAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUr7/JUGwAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBZqtwgAAAAAAACQ==</metadata>
  </media>
  <media bitrate="415" url="2_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_2">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcsgEGJN0vAAFd2lkdGgAQH4AAAAAAAAABmhlaWdodABAcOAAAAAAAAANdmlkZW9kYXRhcmF0ZQBAcDrhxmp71QAJZnJhbWVyYXRlAEA4/42A0ljSAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUr7/JUGwAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBbdPl4AAAAAAACQ==</metadata>
  </media>
  <media bitrate="615" url="3_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_3">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcsgEGJN0vAAFd2lkdGgAQIIAAAAAAAAABmhlaWdodABAdEAAAAAAAAANdmlkZW9kYXRhcmF0ZQBAfLoOxhoCQwAJZnJhbWVyYXRlAEA4/42A0ljSAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUr7/JUGwAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBdhOW0AAAAAAACQ==</metadata>
  </media>
  <media bitrate="955" url="0_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_0">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcsgEGJN0vAAFd2lkdGgAQIYAAAAAAAAABmhlaWdodABAeMAAAAAAAAANdmlkZW9kYXRhcmF0ZQBAiPy3if5LYwAJZnJhbWVyYXRlAEA4/42A0ljSAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUr7/JUGwAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBgSCciAAAAAAACQ==</metadata>
  </media>
  <media bitrate="1655" url="4_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_4">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcsgEGJN0vAAFd2lkdGgAQJAAAAAAAAAABmhlaWdodABAggAAAAAAAAANdmlkZW9kYXRhcmF0ZQBAl27+UGbQvgAJZnJhbWVyYXRlAEA4/42A0ljSAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUr7/JUGwAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBjasJSAAAAAAACQ==</metadata>
  </media>
  <media bitrate="2755" url="5_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_5">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcseuFHrhSAAFd2lkdGgAQJQAAAAAAAAABmhlaWdodABAhoAAAAAAAAANdmlkZW9kYXRhcmF0ZQBApE4NtpueEgAJZnJhbWVyYXRlAEA5AAAAAAAAAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUsjD25OcAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBmK1T6AAAAAAACQ==</metadata>
  </media>
</manifest>

Unfortunately, I think the contents is protected by geographic restrictions, so you may only be able to test this if you are located in Sweden.

[Otto D]

By the way, the "(F4F)" in the original ticket description should really read "(F4M)".

[Otto D]

Replying to cehoyos:

Does FFmpeg support playing the segments / the first transmitted segment?

Oh, I think I misunderstood the question.

I'm not sure if ffmpeg supports playing the first segment / sequence, but I suppose it would. As far as I can understand, the HDS technology is just a way of splitting up the stream and transferring it adaptively in different chunks / bitrates. I think MP4 or flv is the standard container within that stream, but DRM is an optional feature according to the specs which I have ogled.

[Clément Bœsch]

A work in progress is available at ​https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

[Otto D]

Replying to ubitux:

A work in progress is available at ​https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

Interesting.

Would you say that the manifest parsing is complete?

I notice your TODO:

// TODO: make links between bootstraps and medias

What do you mean by this?

[Clément Bœsch]

Replying to ottomatic:

Replying to ubitux:

A work in progress is available at ​https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

Interesting.

Would you say that the manifest parsing is complete?

It should, mostly. The base64 data blob are not parsed, but they are decoded.

I notice your TODO:

// TODO: make links between bootstraps and medias

What do you mean by this?

It just means the bootstrap struct and media struct need to be linked (boostrap->media_id and media->stream_id).

[Otto D]

Replying to ubitux:

Replying to ottomatic:

Replying to ubitux:

A work in progress is available at ​https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

Interesting.

Would you say that the manifest parsing is complete?

It should, mostly. The base64 data blob are not parsed, but they are decoded.

I notice your TODO:

// TODO: make links between bootstraps and medias

What do you mean by this?

It just means the bootstrap struct and media struct need to be linked (boostrap->media_id and media->stream_id).

OK.

So, I'm being new to Git and Github. What is the correct way for me to go about if I want to continue this work and still preserve the awareness of the changes made from the original master code (as well as being able to merge updates from the master into this clone)? Do I just clone your repository, ubitux? And then if I manage to produce some proper code, make a pull request straight to the master?

[Carl Eugen Hoyos]

Replying to ottomatic:

So, I'm being new to Git and Github. What is the correct way for me to go about if I want to continue this work and still preserve the awareness of the changes made from the original master code (as well as being able to merge updates from the master into this clone)? Do I just clone your repository, ubitux? And then if I manage to produce some proper code, make a pull request straight to the master?

If you clone it, all options are still available:
The changes could be merged in the future (that would keep the complete history), or the changes could still be sent as separate patches to ffmpeg-devel.

[Otto D]

I attempted to download the stream at the URL stated in comment 2, using the adobeHDS.php script. That script could not parse the stream as it was encrypted using some sort of Akamai encryption. It would be preferable if that kind of decryption would be supported as well. I would look in the OSMF source for pointers.

[Otto D]

Replying to ubitux:

A work in progress is available at ​https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

Out of curiosity, why did you stall working on this, ubitux?

[Clément Bœsch]

Replying to ottomatic:

Replying to ubitux:

A work in progress is available at ​https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

Out of curiosity, why did you stall working on this, ubitux?

Priorities changed at my company, that technology didn't look appealing, and I don't work for this company anymore so I lost interest.

[Otto D]

Since there has been no further response to this, I have forked ubitux's repository and started working on the hds branch at:
​https://github.com/ottomatic/ffmpeg/tree/hds

I have reached a state where the entire bootstrap structure, with its segment run tables and fragment run tables, is properly parsed.

The progress is a bit slow, but it's progress all the same.

Next, I shall start working on the actual demuxing - if no one else would like to take over?

[Craig Dunford]

Has this been abandoned or is work still ongoing? I'm very interested in this feature and would like to help in some way.

[Otto D]

@cdunford: work us still ongoing. Progress is slow and erratic due to reasons given above, plus I work full time and have two kids to attend to. Any help would be appreciated. First off, properly freeing allocated memory would be good. I'm used to working with managed languages and so I have postponed the whole cleanup bit. Are you used to working with github projects?

[Sunil Deshpande]

Hi Ottomatic,

Just wanted to check, can I use your code from the link ​https://github.com/ottomatic/ffmpeg/tree/hds to test HDS with ffmpeg?

Is it working completely or partially?

Do you have seperate C library for parsing f4m file or HDS?

With Regards,
Sunil

[Otto D]

Replying to sunildeshpande:

Hi Ottomatic,

Just wanted to check, can I use your code from the link ​https://github.com/ottomatic/ffmpeg/tree/hds to test HDS with ffmpeg?

Is it working completely or partially?

It's working partially, to parse the manifest. There is no demuxing of the stream done yet.
It's been a while since I worked on it; other stuff got in the way.

feel free to grab the code and do what you like with it.

Do you have seperate C library for parsing f4m file or HDS?

No, sorry.

With Regards,
Sunil

[Sunil Deshpande]

Hi Ottomatic,

Thanks for your quick reply. I just wanted to know how can I test this partially working stuff. I wanted to test this parsing stuff and then I will try to add the code for demuxing code for the stream. Please let me know how can I test this or how you have tested the parsing code. Can you provide your mail id or chat id for quick discussion?
​deshpande.sunil@gamil.com gmail.com is my mail id. Sicne the things are similar to DASH, we can easyily add the demuxer for the HDS streams also. So please let me know your feedback.

[llogan]

​$15 dololololar bounty offered by hakatashi on Bountysource.

[Otto D]

Just wanted to post an update:

It seems as if there are other people getting closer to a working solution for a demuxer:
​http://ffmpeg.org/pipermail/ffmpeg-devel/2015-May/172628.html

There is a working manifest parser in my Github repo, but the referenced thread on ffmpeg-devel seems to be about a patch which actually performs demuxing. Unfortunately it seems as if the code is only in a patch being sent back and forth, and not in a branch on github.

[gjdfgh]

Unfortunately it seems as if the code is only in a patch being sent back and forth, and not in a branch on github.

That's how development works in FFmpeg. Patches are sent to the mailing list and reviewed there.

Unfortunately, it doesn't look like there was any activity about this patch on the mailing list in the last months.

[Carl Eugen Hoyos]

​http://ffmpeg.org/pipermail/ffmpeg-devel/2015-April/172553.html
​http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/183184/focus=192690