Opened 4 years ago

Last modified 9 months ago

#8684 new defect

WebVTT decoder breaks when ::cue elements are used

Reported by: derrod Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: webvtt
Cc: August Wikerfors Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug: WebVTT input is not read past ::cue tags in input file.
How to reproduce:

% ffmpeg -y -v 9 -loglevel 99 -i test_input.vtt test_output.ass
ffmpeg version git-2020-05-25-6268034 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9.3.1 (GCC) 20200523
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libsrt --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --disable-w32threads --enable-libmfx --enable-ffnvcodec --enable-cuda-llvm --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
  libavutil      56. 49.100 / 56. 49.100
  libavcodec     58. 87.101 / 58. 87.101
  libavformat    58. 43.100 / 58. 43.100
  libavdevice    58.  9.103 / 58.  9.103
  libavfilter     7. 83.100 /  7. 83.100
  libswscale      5.  6.101 /  5.  6.101
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Splitting the commandline.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-i' ... matched as input url with argument 'test_input.vtt'.
Reading option 'test_output.ass' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option y (overwrite output files) with argument 1.
Applying option v (set logging level) with argument 9.
Successfully parsed a group of options.
Parsing a group of options: input url test_input.vtt.
Successfully parsed a group of options.
Opening an input file: test_input.vtt.
[NULL @ 0000025fa3a9ccc0] Opening 'test_input.vtt' for reading
[file @ 0000025fa3a9dd40] Setting default whitelist 'file,crypto,data'
Probing webvtt score:100 size:139
[webvtt @ 0000025fa3a9ccc0] Format webvtt probed with size=2048 and score=100
[webvtt @ 0000025fa3a9ccc0] Before avformat_find_stream_info() pos: 44 bytes read:139 seeks:0 nb_streams:1
[webvtt @ 0000025fa3a9ccc0] All info found
[webvtt @ 0000025fa3a9ccc0] stream 0: start_time: -9223372036854776.000 duration: -9223372036854776.000
[webvtt @ 0000025fa3a9ccc0] format: start_time: -9223372036854.775 duration: -9223372036854.775 (estimate from bit rate) bitrate=0 kb/s
[webvtt @ 0000025fa3a9ccc0] After avformat_find_stream_info() pos: 44 bytes read:139 seeks:0 frames:0
Input #0, webvtt, from 'test_input.vtt':
  Duration: N/A, bitrate: N/A
    Stream #0:0, 0, 1/1000: Subtitle: webvtt
Successfully opened the file.
Parsing a group of options: output url test_output.ass.
Successfully parsed a group of options.
Opening an output file: test_output.ass.
[file @ 0000025fa3aa1b80] Setting default whitelist 'file,crypto,data'
Successfully opened the file.
Output #0, ass, to 'test_output.ass':
  Metadata:
    encoder         : Lavf58.43.100
    Stream #0:0, 0, 1/100: Subtitle: ass (ssa)
    Metadata:
      encoder         : Lavc58.87.101 ssa
Stream mapping:
  Stream #0:0 -> #0:0 (webvtt (native) -> ass (ssa))
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
No more output streams to write to, finishing.
size=       1kB time=00:00:00.00 bitrate=N/A speed=   0x
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:1kB muxing overhead: unknown
Input file #0 (test_input.vtt):
  Input stream #0:0 (subtitle): 0 packets read (0 bytes); 0 frames decoded;
  Total: 0 packets (0 bytes) demuxed
Output file #0 (test_output.ass):
  Output stream #0:0 (subtitle): 0 frames encoded; 0 packets muxed (0 bytes);
  Total: 0 packets (0 bytes) muxed
0 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0000025fa3aa2d40] Statistics: 0 seeks, 1 writeouts
[AVIOContext @ 0000025fa3aa5fc0] Statistics: 139 bytes read, 0 seeks

The entire content of the input file is as follows (also attached):

WEBVTT

::cue(.white){ color: white; }

1
00:00:01.000 --> 00:00:05.000 line:77% position:50% align:middle
<c.white>Hello World</c>

The output file only contains the boilerplate ASS information and no events. (see attached file).

Removing the line(s) starting with ::cue will result in a successfully converted output (albeit without the styling).

Note that even a codec copy from WebVTT to WebVTT will fail and produce an output file that contains only the "WEBVTT" header (other attached file).

Attachments (5)

test_input.vtt (139 bytes ) - added by derrod 4 years ago.
sample input file
test_output.ass (575 bytes ) - added by derrod 4 years ago.
sample ASS output
test_output.vtt (7 bytes ) - added by derrod 4 years ago.
sample stream copy output
test_input.2.vtt (146 bytes ) - added by derrod 4 years ago.
test input with STYLE keyword
example actual svt play subtitle.sv.vtt (101.7 KB ) - added by logv 19 months ago.

Download all attachments as: .zip

Change History (14)

by derrod, 4 years ago

Attachment: test_input.vtt added

sample input file

by derrod, 4 years ago

Attachment: test_output.ass added

sample ASS output

by derrod, 4 years ago

Attachment: test_output.vtt added

sample stream copy output

comment:1 by Carl Eugen Hoyos, 4 years ago

Component: ffmpegavformat

Am I correct that your input file is not valid webvtt?

comment:2 by derrod, 4 years ago

The file was created by an external source and indeed appears to be missing the "STYLE" keyword of the style block (https://www.w3.org/TR/webvtt1/#webvtt-style-block). However, even if the STYLE keyword is added the file is not parsed correctly and the output remains empty.

At least videojs in Chrome still appears to render the output correctly however.

Last edited 4 years ago by derrod (previous) (diff)

by derrod, 4 years ago

Attachment: test_input.2.vtt added

test input with STYLE keyword

comment:3 by Carl Eugen Hoyos, 4 years ago

Priority: minornormal

comment:4 by tpikonen, 23 months ago

comment:6 by logv, 19 months ago

Note to add that this issue currently seems to break processing of all recently produced VTT subtitles served by SVT Play (<https://play.svt.se>). I'm not sure if this is perhaps considered malformed, or if it is something ffmpeg optimally should be able to handle. When trying to convert subtitle format to SRT or ASS, ffmpeg fails silently and produces a 0 byte file (SRT) or just boilerplate (ASS).

E.g., a typical start of a VTT from SVT Play:

WEBVTT

STYLE
::cue {
    color: #DFDFDF;
}
::cue(.huvudpratare) {
    color: #00FFFF;
}
::cue(.ljud) {
    color: #FF00FF;
}
::cue(.ljudskylt) {
    color: #FF00FF;
}
::cue(.relivemark) {
    color: #FF0000;
}
::cue(.sang) {
    color: #00FF00;
}

5da33626-c8a3-48d2-b40a-d5878ea108fa
00:01:34.320 --> 00:01:39.160 align:left position:18%
<c.teletext>Han kommer senare.
Vi kan inte göra nånting åt det nu.</c>

Another (world accessible) example lives here (for a limited time): https://ed4.cdn.svt.se/d0/world/20220811/b9e3c11c-6cdd-46b7-8947-228492d686a5/text/text-0.vtt (also attached)

in reply to:  4 comment:7 by logv, 19 months ago

Replying to tpikonen:

There is patch fixing this issue here: http://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/296353.html

I can verify that this patch seems to fix the problem for me.

comment:8 by varenc, 14 months ago

Any hope of getting that patch to webvttdec merged? I can confirm it fixes this issue!

In the meantime here's a hacky solution that helped me workaround the bug by removing the styling:

 ffmpeg -i <(cat styled-subs.vtt | sd '^STYLE\n::cue\s*\{[^}]*\}' '') out.srt

comment:9 by August Wikerfors, 9 months ago

Cc: August Wikerfors added
Note: See TracTickets for help on using tickets.