Opened 20 months ago
Last modified 10 months ago
#10149 new defect
Unable to read most of HLS M3U8 from Arte TV
Reported by: | Thomas ERNEST | Owned by: | |
---|---|---|---|
Priority: | normal | Component: | avformat |
Version: | 4.4.3 | Keywords: | hls m3u8 fmp4 multivariante |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description (last modified by )
Hello,
I am trying to improve Kodi plugin Arte+7. The improvement aims at reading videos from new Arte TV API instead of "old" Arte HBB TV API. Old content is full mp4 file. New content is HTTP Live Streaming (HLS) packaged in m3u8 files. Some m3u8 files can be read, some cannot.
How to reproduce:
% FAILURE % ./ffplay.exe -v 9 -loglevel 99 https://arte-cmafhls.akamaized.net/am/cmaf/089000/089100/089171-000-A/230120220655/089171-000-A_VF-STF_XQ.m3u8 2> ko.log ... [mov,mp4,m4a,3gp,3g2,mj2 @ 00000228e9e50a00] type:'sidx' parent:'root' sz: 32248 847 9223372036854775807 [mov,mp4,m4a,3gp,3g2,mj2 @ 00000228e9e50a00] sidx reference_type 1 is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented. [mov,mp4,m4a,3gp,3g2,mj2 @ 00000228e9e50a00] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org) [mov,mp4,m4a,3gp,3g2,mj2 @ 00000228e9e50a00] error reading header [AVIOContext @ 00000228e9aae980] Statistics: 35459 bytes read, 0 seeks ... % SUCCESS % ./ffplay.exe -v 9 -loglevel 99 https://arte-cmafhls.akamaized.net/am/cmaf/105000/105600/105610-000-A/230111071325/105610-000-A_VOF-STF_XQ.m3u8 2> ok.log
I reproduced the erroneous and success cases with Kodi 19.5 Matrix and 20.0 Nexus. I reproduced the issue directly with ffplay also and attach the logs of ffplay command here.
EDIT : Both streams have WEBVTT SPRITES. But webvtt for successful playback is not found 404.
Failing one :
- in m3u8 #SPRITES: medias/089171-000-A_SPR.vtt
- deeplink https://arte-cmafhls.akamaized.net/am/cmaf/089000/089100/089171-000-A/230120220655/medias/089171-000-A_SPR.vtt
- content
WEBVTT 00:00:00.000 --> 00:00:06.000 089171-000-A_SPR_000000.jpg#xywh=0,0,240,135 ...
Successful one :
- in m3u8 #SPRITES: medias/105610-000-A_SPR.vtt
- deeplink https://arte-cmafhls.akamaized.net/am/cmaf/105000/105600/105610-000-A/230111071325/medias/105610-000-A_SPR.vtt
- content
Not found
Link to webvtt related tickets :
- https://trac.ffmpeg.org/ticket/5067
- https://trac.ffmpeg.org/ticket/5881
- https://trac.ffmpeg.org/ticket/4048
Thank you in advance
Attachments (15)
Change History (23)
by , 20 months ago
Attachment: | artetv_hls_m3u8_ko.log added |
---|
comment:1 by , 19 months ago
Component: | undetermined → avformat |
---|---|
Description: | modified (diff) |
Keywords: | webvtt added |
Version: | unspecified → 4.4.3 |
by , 19 months ago
Attachment: | failing_089171-000-A_VF-STF_XQ.m3u8 added |
---|
failing m3u8 with hls and webvtt
by , 19 months ago
Attachment: | successful_105610-000-A_VOF-STF_XQ.m3u8 added |
---|
valid m3u8 with hls and webvtt
comment:2 by , 19 months ago
Description: | modified (diff) |
---|
comment:3 by , 19 months ago
Keywords: | fmp4 subtitles added; kodi arte removed |
---|
comment:4 by , 19 months ago
sidx is the direct reason leading to the failure, but it's not the root cause.
- Media initialization section byterange includes sidx. There is no benefical in HLS's case, although DASH can use sidx explicitly. How to interpret the sidx is a question, HLS spec doesn't help. If the byterange excluding sidx, the issue should be gone.
#EXT-X-MAP:URI="089171-000-A_v360.mp4",BYTERANGE="33087@0"
- FFmpeg mp4 demux has no issue to handle the sidx. You can test it with
ffplay 'https://arte-cmafhls.akamaized.net/am/cmaf/089000/089100/089171-000-A/230120220655/medias/089171-000-A_v360.mp4'
The issue happens when playback fmp4 inside HLS. The log message "sidx reference_type 1 is not implemented" is misleading. It's some IO error leads to this error, the fmp4 doesn't use sidx reference_type 1.
I don't have much time to dig further now.
comment:5 by , 18 months ago
there is also that ticket related to webvtt https://trac.ffmpeg.org/ticket/10169
I am able to play such erroneous case by downloading separately video mp4 stream, audio mp4 stream, and webvtt (by following each m3u8) (removing any STYLE blocks in webvtt like said in #10169) and combine/merge them into a file; for local reading.
comment:6 by , 12 months ago
Keywords: | multivariante added; webvtt subtitles removed |
---|
Indeed the problem is not related to webvtt or subtitles. The root cause is related to the multivariant playlist https://developer.apple.com/documentation/http-live-streaming/creating-a-multivariant-playlist or byterange addressing.
It is possible to build a set of new m3u to index streams without multivariant and byterange addressing so that FFMPEG can play it. Nevertheless, there is still a problem with these new m3u : the content is not seekable. It is not possible to fast-forward, rewind or jump to a moment in video.
To reproduce:
- GET https://www.arte.tv/api/rproxy/emac/v4/fr/tv/pages/HOME/
- pick any programId from the response e.g. 020528-000-A
- GET https://api.arte.tv/api/player/v2/config/fr/020528-000-A
- Read URLs to m3u in data.attributes.streams.url
Notes : Each m3u has different combinaition audio and subtibles e.g. original version with french subtitles, german language without subtitles... m3u have the same multivariant video streams. Different audio stream and subtitles combinaison.
- Pick a video stream and every audios streams. Create a m3u to encapsulate each stream.
- Create a parent m3u to index and add metadata to stream m3u.
Notes : See m3u attached to the ticket as an example. m3u from step 5 and 6 follow the pattern in https://developer.apple.com/documentation/http-live-streaming/adding-alternate-media-to-a-playlist.
by , 12 months ago
Attachment: | 020528-000-A_de.m3u8 added |
---|
german audio stream m3u for program 20528
by , 12 months ago
Attachment: | 020528-000-A_en.m3u8 added |
---|
english audio stream m3u for program 20528
by , 12 months ago
Attachment: | 020528-000-A_fr.m3u8 added |
---|
french audio stream m3u for program 20528
by , 12 months ago
Attachment: | ffplay_homemade_multi_lang_m3u.log added |
---|
ffplay logs playing 20528 m3u set - not seekable
comment:7 by , 12 months ago
If I add in the m3u8 files #EXT-X-TARGETDURATION:6360 - 6360 being content duration in seconds - then I have the ability to seek the playlist with ffplay 6.0, but not with Kodi 20.1 (integrating ffplay 4.4.1 as far as I recall).
- #EXT-X-TARGETDURATION:6360
Please find in attachements a set of m3u8 which I can navigate through.
I added other fields after a first successfull test as far as I remember. I guess they don't influence ffplay to seek or not the playlist :
- #EXT-X-PLAYLIST-TYPE:VOD
- #EXT-X-MEDIA-SEQUENCE:0
- #EXT-X-ENDLIST
by , 12 months ago
Attachment: | 020528-000-A_en.2.m3u8 added |
---|
seekable example - english audio stream
by , 12 months ago
Attachment: | 020528-000-A_main - Copie.m3u8 added |
---|
seekable example - main content index
comment:8 by , 11 months ago
I have been getting this error on some of my HLS streams. I noticed it only happens on streams that are longer than around 8000 seconds (or 2666 segments).
Initially I thought this could be an issue with unsigned 32-bit integer overflow of the byterange addresses, but it also happens on the lighter variants such as 432p which don't have addresses that overflow a 32bit unsigned int.
Logs of failing case