Opened 4 years ago

Last modified 5 months ago

#1964 open enhancement

Request support for decoding / demuxing Adobe HDS dynamic http streaming

Reported by: ottomatic Owned by:
Priority: wish Component: avformat
Version: git-master Keywords: HDS bounty xml
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Adobe HDS is an adaptive streaming format used primarily to deliver video streams through content delivery networks.

It uses a manifest file (F4F) to describe the segments of a file, and then it adaptively delivers segments and sequences in the "optimal" bitrate depending on the client's bandwith and the total server load.

Certain major content providers are moving to this format (at least in Sweden) and it would be really great if ffmpeg could support it.

A good summary of HDS is found at:
http://rdkls.blogspot.se/2011/11/what-i-know-about-http-adaptive.html

It seems that most projects that do decode such streams use a PHP script (!) found at:
https://github.com/K-S-V/Scripts/blob/master/AdobeHDS.php

The Open Source Media Player project has action script code which probably does the same thing:
http://sourceforge.net/adobe/osmf/

The format is quite similar to applehttp/hls, so it should be possible to borrow some patterns from the support for that format, which is already in libavformat.

I am a C# / java developer and could probably make this happen in C, but it would take a lot of effort. I am not yet familiar with the ffmpeg source code. So I'm hoping that someone might already be working on this? Or that someone well versed on avformat development can take this on. I'd be happy to contribute with my own efforts.

Best regards!

Change History (23)

comment:1 follow-up: Changed 4 years ago by cehoyos

  • Keywords Adobe removed

Does FFmpeg support playing the segments / the first transmitted segment?

comment:2 Changed 4 years ago by ottomatic

No.

Here is the output from when I attempt to play a stream, using the recent static build for 64 bit windows:

>ffmpeg.exe -i "http://svtplay6s-f.akamaihd.net/z/se
/secure/20121118/1232796-008A/EARTHLY_TREASUR-008A-7a4e5968ffc1c170_,900,320,420
,620,1660,2760,.mp4.csmil/manifest.f4m?hdcore=2.8.0&g=LLIOFRVAXRWA" D:\tmp\fmt-t
est.ts
ffmpeg version N-46469-gc995644 Copyright (c) 2000-2012 the FFmpeg developers
  built on Nov  5 2012 17:58:15 with gcc 4.7.2 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-pthreads --enable-runt
ime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libass -
-enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enab
le-libgsm --enable-libmp3lame --enable-libnut --enable-libopenjpeg --enable-libo
pus --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheo
ra --enable-libutvideo --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-li
bvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --ena
ble-zlib
  libavutil      52.  5.100 / 52.  5.100
  libavcodec     54. 71.100 / 54. 71.100
  libavformat    54. 36.100 / 54. 36.100
  libavdevice    54.  3.100 / 54.  3.100
  libavfilter     3. 21.106 /  3. 21.106
  libswscale      2.  1.102 /  2.  1.102
  libswresample   0. 16.100 /  0. 16.100
  libpostproc    52.  1.100 / 52.  1.100
http://svtplay6s-f.akamaihd.net/z/se/secure/20121118/1232796-008A/EARTHLY_TREASU
R-008A-7a4e5968ffc1c170_,900,320,420,620,1660,2760,.mp4.csmil/manifest.f4m?hdcor
e=2.8.0&g=LLIOFRVAXRWA: Invalid data found when processing input

If I inspect the contents of that URL, I get the F4M manifest:

<?xml version="1.0" encoding="UTF-8"?>
<manifest xmlns="http://ns.adobe.com/f4m/1.0" xmlns:akamai="uri:akamai.com/f4m/1.0">
  <akamai:version>2.0</akamai:version>
  <akamai:bw>5000</akamai:bw>
  <id>/se/secure/20121118/1232796-008A/EARTHLY_TREASUR-008A-7a4e5968ffc1c170_,900,320,420,620,1660,2760,.mp4.csmil_0</id>
  <streamType>recorded</streamType>
  <akamai:streamType>vod</akamai:streamType>
  <duration>300.501</duration>
  <streamBaseTime>0.000</streamBaseTime>
  <bootstrapInfo profile="named" id="bootstrap_1">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEldUAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGWUAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <bootstrapInfo profile="named" id="bootstrap_2">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEldUAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGWUAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <bootstrapInfo profile="named" id="bootstrap_3">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEldUAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGWUAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <bootstrapInfo profile="named" id="bootstrap_0">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEldUAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGWUAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <bootstrapInfo profile="named" id="bootstrap_4">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEldUAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGWUAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <bootstrapInfo profile="named" id="bootstrap_5">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAElcAAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGVAAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <media bitrate="315" url="1_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_1">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcsgEGJN0vAAFd2lkdGgAQHQAAAAAAAAABmhlaWdodABAZoAAAAAAAAANdmlkZW9kYXRhcmF0ZQBAY/imU7h9mQAJZnJhbWVyYXRlAEA4/42A0ljSAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUr7/JUGwAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBZqtwgAAAAAAACQ==</metadata>
  </media>
  <media bitrate="415" url="2_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_2">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcsgEGJN0vAAFd2lkdGgAQH4AAAAAAAAABmhlaWdodABAcOAAAAAAAAANdmlkZW9kYXRhcmF0ZQBAcDrhxmp71QAJZnJhbWVyYXRlAEA4/42A0ljSAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUr7/JUGwAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBbdPl4AAAAAAACQ==</metadata>
  </media>
  <media bitrate="615" url="3_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_3">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcsgEGJN0vAAFd2lkdGgAQIIAAAAAAAAABmhlaWdodABAdEAAAAAAAAANdmlkZW9kYXRhcmF0ZQBAfLoOxhoCQwAJZnJhbWVyYXRlAEA4/42A0ljSAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUr7/JUGwAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBdhOW0AAAAAAACQ==</metadata>
  </media>
  <media bitrate="955" url="0_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_0">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcsgEGJN0vAAFd2lkdGgAQIYAAAAAAAAABmhlaWdodABAeMAAAAAAAAANdmlkZW9kYXRhcmF0ZQBAiPy3if5LYwAJZnJhbWVyYXRlAEA4/42A0ljSAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUr7/JUGwAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBgSCciAAAAAAACQ==</metadata>
  </media>
  <media bitrate="1655" url="4_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_4">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcsgEGJN0vAAFd2lkdGgAQJAAAAAAAAAABmhlaWdodABAggAAAAAAAAANdmlkZW9kYXRhcmF0ZQBAl27+UGbQvgAJZnJhbWVyYXRlAEA4/42A0ljSAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUr7/JUGwAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBjasJSAAAAAAACQ==</metadata>
  </media>
  <media bitrate="2755" url="5_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_5">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcseuFHrhSAAFd2lkdGgAQJQAAAAAAAAABmhlaWdodABAhoAAAAAAAAANdmlkZW9kYXRhcmF0ZQBApE4NtpueEgAJZnJhbWVyYXRlAEA5AAAAAAAAAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUsjD25OcAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBmK1T6AAAAAAACQ==</metadata>
  </media>
</manifest>

Unfortunately, I think the contents is protected by geographic restrictions, so you may only be able to test this if you are located in Sweden.

Last edited 4 years ago by ottomatic (previous) (diff)

comment:3 Changed 4 years ago by ottomatic

By the way, the "(F4F)" in the original ticket description should really read "(F4M)".

comment:4 in reply to: ↑ 1 Changed 4 years ago by ottomatic

Replying to cehoyos:

Does FFmpeg support playing the segments / the first transmitted segment?

Oh, I think I misunderstood the question.

I'm not sure if ffmpeg supports playing the first segment / sequence, but I suppose it would. As far as I can understand, the HDS technology is just a way of splitting up the stream and transferring it adaptively in different chunks / bitrates. I think MP4 or flv is the standard container within that stream, but DRM is an optional feature according to the specs which I have ogled.

comment:5 follow-ups: Changed 4 years ago by ubitux

A work in progress is available at https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

comment:6 in reply to: ↑ 5 ; follow-up: Changed 4 years ago by ottomatic

Replying to ubitux:

A work in progress is available at https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

Interesting.

Would you say that the manifest parsing is complete?

I notice your TODO:

// TODO: make links between bootstraps and medias

What do you mean by this?

comment:7 in reply to: ↑ 6 ; follow-up: Changed 4 years ago by ubitux

Replying to ottomatic:

Replying to ubitux:

A work in progress is available at https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

Interesting.

Would you say that the manifest parsing is complete?

It should, mostly. The base64 data blob are not parsed, but they are decoded.

I notice your TODO:

// TODO: make links between bootstraps and medias

What do you mean by this?

It just means the bootstrap struct and media struct need to be linked (boostrap->media_id and media->stream_id).

comment:8 in reply to: ↑ 7 ; follow-up: Changed 4 years ago by ottomatic

Replying to ubitux:

Replying to ottomatic:

Replying to ubitux:

A work in progress is available at https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

Interesting.

Would you say that the manifest parsing is complete?

It should, mostly. The base64 data blob are not parsed, but they are decoded.

I notice your TODO:

// TODO: make links between bootstraps and medias

What do you mean by this?

It just means the bootstrap struct and media struct need to be linked (boostrap->media_id and media->stream_id).

OK.

So, I'm being new to Git and Github. What is the correct way for me to go about if I want to continue this work and still preserve the awareness of the changes made from the original master code (as well as being able to merge updates from the master into this clone)? Do I just clone your repository, ubitux? And then if I manage to produce some proper code, make a pull request straight to the master?

comment:9 in reply to: ↑ 8 Changed 4 years ago by cehoyos

Replying to ottomatic:

So, I'm being new to Git and Github. What is the correct way for me to go about if I want to continue this work and still preserve the awareness of the changes made from the original master code (as well as being able to merge updates from the master into this clone)? Do I just clone your repository, ubitux? And then if I manage to produce some proper code, make a pull request straight to the master?

If you clone it, all options are still available:
The changes could be merged in the future (that would keep the complete history), or the changes could still be sent as separate patches to ffmpeg-devel.

comment:10 Changed 4 years ago by cehoyos

  • Priority changed from normal to wish
  • Status changed from new to open
  • Version changed from unspecified to git-master

comment:11 Changed 4 years ago by ottomatic

I attempted to download the stream at the URL stated in comment 2, using the adobeHDS.php script. That script could not parse the stream as it was encrypted using some sort of Akamai encryption. It would be preferable if that kind of decryption would be supported as well. I would look in the OSMF source for pointers.

Last edited 4 years ago by ottomatic (previous) (diff)

comment:12 in reply to: ↑ 5 ; follow-up: Changed 4 years ago by ottomatic

Replying to ubitux:

A work in progress is available at https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

Out of curiosity, why did you stall working on this, ubitux?

comment:13 in reply to: ↑ 12 Changed 4 years ago by ubitux

Replying to ottomatic:

Replying to ubitux:

A work in progress is available at https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

Out of curiosity, why did you stall working on this, ubitux?

Priorities changed at my company, that technology didn't look appealing, and I don't work for this company anymore so I lost interest.

comment:14 Changed 3 years ago by ottomatic

Since there has been no further response to this, I have forked ubitux's repository and started working on the hds branch at:
https://github.com/ottomatic/ffmpeg/tree/hds

I have reached a state where the entire bootstrap structure, with its segment run tables and fragment run tables, is properly parsed.

The progress is a bit slow, but it's progress all the same.

Next, I shall start working on the actual demuxing - if no one else would like to take over?

comment:15 Changed 3 years ago by cdunford

Has this been abandoned or is work still ongoing? I'm very interested in this feature and would like to help in some way.

comment:16 Changed 3 years ago by ottomatic

@cdunford: work us still ongoing. Progress is slow and erratic due to reasons given above, plus I work full time and have two kids to attend to. Any help would be appreciated. First off, properly freeing allocated memory would be good. I'm used to working with managed languages and so I have postponed the whole cleanup bit. Are you used to working with github projects?

Last edited 3 years ago by ottomatic (previous) (diff)

comment:17 follow-up: Changed 3 years ago by sunildeshpande

Hi Ottomatic,

Just wanted to check, can I use your code from the link https://github.com/ottomatic/ffmpeg/tree/hds to test HDS with ffmpeg?

Is it working completely or partially?

Do you have seperate C library for parsing f4m file or HDS?

With Regards,
Sunil

comment:18 in reply to: ↑ 17 Changed 3 years ago by ottomatic

Replying to sunildeshpande:

Hi Ottomatic,

Just wanted to check, can I use your code from the link https://github.com/ottomatic/ffmpeg/tree/hds to test HDS with ffmpeg?

Is it working completely or partially?

It's working partially, to parse the manifest. There is no demuxing of the stream done yet.
It's been a while since I worked on it; other stuff got in the way.

feel free to grab the code and do what you like with it.

Do you have seperate C library for parsing f4m file or HDS?

No, sorry.

With Regards,
Sunil

comment:19 Changed 3 years ago by sunildeshpande

Hi Ottomatic,

Thanks for your quick reply. I just wanted to know how can I test this partially working stuff. I wanted to test this parsing stuff and then I will try to add the code for demuxing code for the stream. Please let me know how can I test this or how you have tested the parsing code. Can you provide your mail id or chat id for quick discussion?
deshpande.sunil@gamil.com gmail.com is my mail id. Sicne the things are similar to DASH, we can easyily add the demuxer for the HDS streams also. So please let me know your feedback.

comment:21 Changed 12 months ago by ottomatic

Just wanted to post an update:

It seems as if there are other people getting closer to a working solution for a demuxer:
http://ffmpeg.org/pipermail/ffmpeg-devel/2015-May/172628.html

There is a working manifest parser in my Github repo, but the referenced thread on ffmpeg-devel seems to be about a patch which actually performs demuxing. Unfortunately it seems as if the code is only in a patch being sent back and forth, and not in a branch on github.

comment:22 Changed 12 months ago by gjdfgh

Unfortunately it seems as if the code is only in a patch being sent back and forth, and not in a branch on github.

That's how development works in FFmpeg. Patches are sent to the mailing list and reviewed there.

Unfortunately, it doesn't look like there was any activity about this patch on the mailing list in the last months.

Note: See TracTickets for help on using tickets.