Opened 2 years ago
Last modified 2 years ago
#9979 new enhancement
webvtt subtitles not properly detected in DASH manifest
Reported by: | cookieguru | Owned by: | |
---|---|---|---|
Priority: | wish | Component: | avformat |
Version: | git-master | Keywords: | dash webvtt |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
Summary of the bug:
If an MPD manifest contains WebVTT subtitles FFmpeg will not properly recognize or decode them.
How to reproduce:
Given a MPD with webvtt files, ffmpeg does not recognize it as a subtitle stream and instead labels it as an unknown data stream:
ffprobe -hide_banner "https://www.example.com/master.mpd"
Input #0, dash, from 'https://www.example.com/master.mpd': Duration: 00:42:00.00, start: 0.000000, bitrate: 0 kb/s Program 0 Stream #0:0: Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 416x234 [SAR 1:1 DAR 16:9], 162 kb/s, 29.97 fps, 29.97 tbr, 60k tbn (default) Metadata: variant_bitrate : 139000 id : video=139000 (snip: more video streams) Stream #0:8(en): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 126 kb/s (default) Metadata: variant_bitrate : 125000 id : audio_eng=125000 Stream #0:9(en): Data: none (wvtt / 0x74747677), 0 kb/s (default) Metadata: id : 1 Unsupported codec with id 0 for input stream 9
In this example, stream 9 contains the subtitles but is marked as Data.
Here is the contents of the MPD file:
<?xml version="1.0"?> <MPD xmlns="urn:mpeg:dash:schema:mpd:2011" profiles="urn:mpeg:dash:profile:isoff-live:2011,urn:com:dashif:dash264" type="static" mediaPresentationDuration="PT42M0.818300S" minBufferTime="PT10S"> <Period id="1" duration="PT42M0.8183S"> <!-- audio and video AdaptationSet removed for brevity --> <AdaptationSet mimeType="application/mp4" codecs="wvtt" startWithSAP="1" segmentAlignment="true" lang="en" contentType="text"> <Role schemeIdUri="urn:mpeg:dash:role:2011" value="subtitle"/> <SegmentTemplate initialization="caption_uuid_init.mp4" media="caption_uuid_$Time$.m4s" startNumber="1" timescale="1000"> <SegmentTimeline> <S t="0" d="30000" r="15"/> <S d="27273"/> <S d="30000" r="16"/> <S d="21498"/> <S d="30000" r="9"/> <S d="7941"/> <S d="30000" r="9"/> <S d="967"/> <S d="30000" r="17"/> <S d="15688"/> <S d="30000" r="9"/> <S d="9733"/> </SegmentTimeline> </SegmentTemplate> <Representation mimeType="application/mp4" bandwidth="256" codecs="wvtt" id="1"/> </AdaptationSet> <EventStream schemeIdUri="urn:scte:scte35:2014:xml+bin" timescale="30000"> <Event id="1" presentationTime="15218203" duration="0"/> <Event id="2" presentationTime="31163132" duration="0"/> <Event id="3" presentationTime="40401361" duration="0"/> <Event id="4" presentationTime="49430381" duration="0"/> <Event id="5" presentationTime="66101035" duration="0"/> </EventStream> </Period> </MPD>
Note codecs="wvtt"
located twice in this AdaptationSet
.
If we manually download the contents of this stream:
ffmpeg -hide_banner -i "https://www.example.com/master.mpd" -map 0:d -c copy -copy_unknown -f data raw.bin
We can see that it contains the subs, but note it is not a valid webVTT file (first byte is a NUL byte):
xxd raw.bin | head 00000000: 0000 005f 7674 7463 0000 0009 6964 656e ..._vttc....iden 00000010: 3800 0000 4e70 6179 6c3c 693e 5b6e 6172 8...Npayl<i>[nar 00000020: 7261 746f 725d 2041 6e64 2061 2066 616e rator] And a fan 00000030: 7461 7374 6963 616c 3c2f 693e 0a3c 693e tastical</i>.<i> 00000040: 776f 726c 6420 6869 6464 656e 2069 6e20 world hidden in 00000050: 7468 6520 6a75 6e67 6c65 2e3c 2f69 3e00 the jungle.</i>. 00000060: 0000 5f76 7474 6300 0000 0969 6465 6e38 .._vttc....iden8 00000070: 0000 004e 7061 796c 3c69 3e5b 6e61 7272 ...Npayl<i>[narr 00000080: 6174 6f72 5d20 416e 6420 6120 6661 6e74 ator] And a fant 00000090: 6173 7469 6361 6c3c 2f69 3e0a 3c69 3e77 astical</i>.<i>w
The URL of the stream in question is time-limited (it's a signed URL with an expiration parameter) and thus is not included here. US-based users can probably find similar MPD files on https://go.discovery.com/ but note that the site uses both HLS and DASH streaming (newer videos tend to be DASH; older videos never are).
ffmpeg version:
ffmpeg version N-108711-g3141dbb7ad-20221018 Copyright (c) 2000-2022 the FFmpeg developers built with gcc 12.1.0 (crosstool-NG 1.25.0.55_3defb7b) configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20221018 libavutil 57. 39.101 / 57. 39.101 libavcodec 59. 51.100 / 59. 51.100 libavformat 59. 34.101 / 59. 34.101 libavdevice 59. 8.101 / 59. 8.101 libavfilter 8. 49.101 / 8. 49.101 libswscale 6. 8.112 / 6. 8.112 libswresample 4. 9.100 / 4. 9.100 libpostproc 56. 7.100 / 56. 7.100
Attachments (1)
Change History (4)
comment:1 by , 2 years ago
Keywords: | mpd added; mp removed |
---|
comment:3 by , 2 years ago
Component: | undetermined → avformat |
---|---|
Keywords: | DASH dashdec mpd removed |
Priority: | normal → wish |
Type: | defect → enhancement |
by , 2 years ago
raw vtt stream obtained from https://go.discovery.com/video/deadliest-catch-discovery/greenhorns
Possibly related: #9839