Last subtitle in MP4 is displayed forever

[fumoboy007]

Summary of the bug:
Decoding the last tx3g subtitle in an MP4 produces an incorrect duration, causing the subtitle to stay on-screen until the end of the media.

How to reproduce:
Play the attached file in VLC. Also, run the following command and compare the outputted subtitle file with the original subtitle file (also attached).

$ ffmpeg -i Permanent\ Last\ Subtitle.mp4 -map 0:s:0 FFmpeg\ Output.srt
ffmpeg version 4.1 Copyright (c) 2000-2018 the FFmpeg developers
  built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1_1 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gpl --enable-libmp3lame --enable-libopus --enable-libsnappy --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-librtmp --enable-libspeex --enable-videotoolbox
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Permanent Last Subtitle.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: mp41mp42isom
    creation_time   : 2019-04-18T05:09:42.000000Z
  Duration: 00:01:05.04, start: 0.000000, bitrate: 431 kb/s
    Stream #0:0(deu): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, smpte170m), 640x360, 289 kb/s, 23.97 fps, 23.98 tbr, 24k tbn, 48k tbc (default)
    Metadata:
      creation_time   : 2019-04-18T05:09:42.000000Z
      handler_name    : Core Media Video
      encoder         : AVC Coding
    Stream #0:1(deu): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 137 kb/s (default)
    Metadata:
      creation_time   : 2019-04-18T05:09:42.000000Z
      handler_name    : Core Media Audio
    Stream #0:2(eng): Subtitle: mov_text (tx3g / 0x67337874), 640x54, 0 kb/s (default)
    Metadata:
      rotate          : 0
      creation_time   : 2019-04-18T05:10:26.000000Z
Output #0, srt, to 'FFmpeg Output.srt':
  Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: mp41mp42isom
    encoder         : Lavf58.20.100
    Stream #0:0(eng): Subtitle: subrip (srt), 640x54 (default)
    Metadata:
      encoder         : Lavc58.35.100 srt
      creation_time   : 2019-04-18T05:10:26.000000Z
Stream mapping:
  Stream #0:2 -> #0:0 (mov_text (native) -> subrip (srt))
Press [q] to stop, [?] for help
size=       0kB time=00:00:45.25 bitrate=   0.0kbits/s speed=3.59e+04x    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 79.069771%

[fumoboy007]

This is reproducible with any MP4+tx3g file. I’m surprised this bug made it through!

[Carl Eugen Hoyos]

Please test current FFmpeg git head to make this a valid ticket (there is no release support on this bug tracker) and please suppress your comments if you want a developer to look at your tickets.
Do not attach output files unless specifically asked to.

[mkver]

Duplicate of #6341.

[fumoboy007]

Replying to cehoyos:

Please test current FFmpeg git head to make this a valid ticket (there is no release support on this bug tracker)

This policy does not make sense in this context where I have submitted a test file and command to reproduce the problem. I have my own commits on top FFmpeg; rebasing onto master and rebuilding takes a lot of time. I have given you exactly what you need to reproduce the problem; you can do so using the latest commit if you so desire.

and please suppress your comments if you want a developer to look at your tickets.

I don’t understand what you mean by “suppress your comments”.

Do not attach output files unless specifically asked to.

Please tell me you’re joking…

[mkver]

1. My earlier claim was wrong. This is not a duplicate of #6341.
2. In fact, #6341 was fixed some time ago.
3. The differences between interm.mp4 from #6341 and this sample are as follows:
   a) interm.mp4 has an empty packet at timestamp 0 and empty packets to end both subtitle lines; so there are five packets and dts deltas are 3s, 10s, 2s, 2s and 0. This means that the packet's dts values are 0s (the first sample always has dts 0 (before edit lists are applied), 3s, 13s, 15s, 17s, which means that the actual subtitle lines timestamps are from 3s-13s and 15s-17s. (I don't know if the last dts delta entry is necessary or not.)
   b) The sample from this issue is different: It has two samples (for one line of subtitles) and two entries in the stts containing two dts deltas: 5s and 3s, so that the dts of the first sample (the "empty" sample) is 0s and the time of the second sample (containing the actual subtitle) is 5s. And if there were another sample, then this sample would have a dts of 8s. But there isn't. Furthermore, the track as a whole has a duration of 8s. I am not sure whether this file is completely spec-compliant; I am leaning to yes.
4. Even if this file isn't spec-compliant, one could nevertheless support it. In this case this ticket might become a "wish". But I'll leave it as defect for the time being.
5. Modified versions of FFmpeg are even more unsupported than release versions. So you actually should not rebase your own commits on current git head, but just test current git head.
   The rationale for this policy is that it would waste developer's time to test something that might already have been fixed.
6. The comment cehoyos wants you to suppress is surely "I’m surprised this bug made it through!". The claim you base this on is btw wrong as the sample from #6341 shows.

[fumoboy007]

Replying to mkver:

b) The sample from this issue is different: It has two samples (for one line of subtitles) and two entries in the stts containing two dts deltas: 5s and 3s, so that the dts of the first sample (the "empty" sample) is 0s and the time of the second sample (containing the actual subtitle) is 5s. And if there were another sample, then this sample would have a dts of 8s. But there isn't. Furthermore, the track as a whole has a duration of 8s. I am not sure whether this file is completely spec-compliant; I am leaning to yes.

I would like to help debug this, but I have no knowledge of the format. Is there somewhere I can freely download the MPEG-4 Timed Text specification (is that the right one?) or is there some document that summarizes the specification?

5. Modified versions of FFmpeg are even more unsupported than release versions. So you actually should not rebase your own commits on current git head, but just test current git head.
   The rationale for this policy is that it would waste developer's time to test something that might already have been fixed.

Oh, I see git head is more than a thousand commits ahead of the releases. OK, I will spend the extra time to test on git head for future bug reports in case the issue is already fixed.

6. The comment cehoyos wants you to suppress is surely "I’m surprised this bug made it through!". The claim you base this on is btw wrong as the sample from #6341 shows.

Sorry for the false claim. My “surprised” comment was not meant to be accusatory; sorry if it came off that way. That being said, it would have been more productive to post a concrete rejection of the claim rather than a passive-aggressive response.

[mkver]

The mp4 file format is described in ISO/IEC 14496-12 ("ISO base media file format"); timed text is described in ISO/IEC 14496-17 ("Streaming text format").

[mkver]

ISO/IEC 14496-17 is absolutely uninformative for this question:
"[T]o each text access unit a single time stamp applies. The time stamp assigned to a text access unit indicates the time at which the text access unit is to be presented on the display."

And the reason for this is that the carriage of timed text in mp4 is actually described in 14496-30, a document which doesn't seem to be available for free.

[fumoboy007]

Replying to mkver:

ISO/IEC 14496-17 is absolutely uninformative for this question:
"[T]o each text access unit a single time stamp applies. The time stamp assigned to a text access unit indicates the time at which the text access unit is to be presented on the display."

And the reason for this is that the carriage of timed text in mp4 is actually described in 14496-30, a document which doesn't seem to be available for free.

Ah! Does this ​draft have enough information?

[fumoboy007]

I submitted a ​patch that fixes this.

[Gregory Beauregard]

Replying to fumoboy007:

I submitted a ​patch that fixes this.

This issue still exists in ffmpeg 5.0; did you ever post a followup patch after the comments on fate breaking? I couldn't find one on the ML.

[Gregory Beauregard]

As an aside did you find any workaround for this? -fix_sub_duration when converting to another format deletes the last subtitle line entirely; converting to ass/srt retains the long length of the last line. mpv is rendering it okay so maybe they have figured out a workaround?

[Balling]

WEll, you do not even apply editlist duration for aac audio! Ha. #7828

[Balling]

VLC has this bug indeed, on I.Am.Groot.S01E03.2160p.DSNP.WEB-DL.DDP5.1.Atmos.DV.MP4.x265-DVSUX.mp4 the last subtitle continues until the very end. HAHA.

While in mpv it or even Permanent\ Last\ Subtitle.mp4 vanishes very soon. Just great.

[Balling]

This got fixed in TsMuxer, so ffmpeg now is really bad. ​https://github.com/justdan96/tsMuxer/issues/673

Subtitle Edit 3.6.12 also has no such issue.