webvtt not seeking properly

[Tristan Matthews]

Summary of the bug:

When trimming and muxing a video with accompanying VTT file (using output seeking as per ​https://trac.ffmpeg.org/wiki/Seeking#Output), if I specify -c:s webvttthe subtitles are not present as expected in the output.

However, if I use -c:s copy instead, the subtitles are present and line up as expected.

How to reproduce:

Download ​https://people.videolan.org/~tmatth/samples/counting.mp4 and ​https://people.videolan.org/~tmatth/samples/counting.vtt

and trim and mux them with:

% ffmpeg -i counting.mp4 -i counting.vtt -shortest -c:v copy -c:s webvtt -ss 20s trimmed.mkv

To see the working/expected version, do:

% ffmpeg -i counting.mp4 -i counting.vtt -shortest -c:v copy -c:s copy -ss 20s trimmed.mkv

It's possible this should be filed as an "enhancement".

[Tristan Matthews]

debug level output from running the "-c:s webvtt" encoding

[MasterQuestionable]

͏    So what went wrong exactly..?
͏    Subtitle stream outright went missing with codec "webvtt" specified?

͏    Note "-shortest" means to terminate encoding upon the end of the shortest input stream.
͏    "copy" codec means to outright copy without re-encoding: which might workaround the restriction.

[Tristan Matthews]

The subtitles don't display at the expected times unless I use -c:s copy. The subtitle track is present according to ffprobe however:

# when using `-c:s webvtt

Input #0, matroska,webm, from 'trimmed.mkv':
  Metadata:
    COMPATIBLE_BRANDS: isomiso6iso2avc1mp41
    MAJOR_BRAND     : isom
    MINOR_VERSION   : 512
    ENCODER         : Lavf61.5.101
  Duration: 00:00:13.75, start: -0.005000, bitrate: 2212 kb/s
  Stream #0:0(eng): Video: h264 (Constrained Baseline), yuv420p(tv, bt709, progressive), 1920x1080, SAR 1:1 DAR 16:9, 59.71 fps, 59.71 tbr, 1k tbn (default)
      Metadata:
        HANDLER_NAME    : VideoHandler
        VENDOR_ID       : [0][0][0][0]
        DURATION        : 00:00:13.748000000
  Stream #0:1(eng): Audio: ac3, 48000 Hz, stereo, fltp, 192 kb/s (default)
      Metadata:
        HANDLER_NAME    : SoundHandler
        VENDOR_ID       : [0][0][0][0]
        ENCODER         : Lavc61.10.100 ac3
        DURATION        : 00:00:13.728000000
  Stream #0:2: Subtitle: webvtt
      Metadata:
        ENCODER         : Lavc61.10.100 webvtt
        DURATION        : 00:00:00.000000000

# when using -c:s copy

Input #0, matroska,webm, from 'trimmed.mkv':
  Metadata:
    COMPATIBLE_BRANDS: isomiso6iso2avc1mp41
    MAJOR_BRAND     : isom
    MINOR_VERSION   : 512
    ENCODER         : Lavf61.5.101
  Duration: 00:00:13.76, start: -0.005000, bitrate: 2210 kb/s
  Stream #0:0(eng): Video: h264 (Constrained Baseline), yuv420p(tv, bt709, progressive), 1920x1080, SAR 1:1 DAR 16:9, 59.71 fps, 59.71 tbr, 1k tbn (default)
      Metadata:
        HANDLER_NAME    : VideoHandler
        VENDOR_ID       : [0][0][0][0]
        DURATION        : 00:00:13.748000000
  Stream #0:1(eng): Audio: ac3, 48000 Hz, stereo, fltp, 192 kb/s (default)
      Metadata:
        HANDLER_NAME    : SoundHandler
        VENDOR_ID       : [0][0][0][0]
        ENCODER         : Lavc61.10.100 ac3
        DURATION        : 00:00:13.760000000
  Stream #0:2: Subtitle: webvtt
      Metadata:
        DURATION        : 00:00:13.720000000

The issue is that I wouldn't expect the webvtt codec to effectively nullify the subtitles in this instance.

[MasterQuestionable]

͏    Present indeed, but I suppose the output subtitle is empty. (during of 0?)

͏    Is it relevant with "-shortest"?

[Tristan Matthews]

I wouldn't get hung up over the use of "-shortest" or not, if omitted it just means the whole subtitle track (untruncated) will be present in the container, but still not correctly aligned in time (and thus never rendered).

The bug (or missing enhancement) AFAICT is around output seeking with the webvtt codec.

[MasterQuestionable]

͏    comment:1 has explained the potential implications.

͏    You sure not using "-shortest" with "-ss 20" would cause the inclusion of the whole "counting.vtt"?
[[

WEBVTT

00:00:02.680 --> 00:00:13.760
 12, one,

00:00:13.760 --> 00:00:24.040
 two, three,

00:00:24.040 --> 00:00:29.920
 four, five.

00:00:33.200 --> 00:00:33.720
 Six.

]]