Opened 8 years ago

Last modified 6 years ago

#5756 open defect

Robustness issue with nul character in ASS subtitle streams

Reported by: Thierry Lelegard Owned by:
Priority: normal Component: undetermined
Version: git-master Keywords: ass mkv
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

This report describes a robustness issue with the processing of ASS subtitles.

ffmpeg version N-81256-gd3426fb (full text in commands below)

Summary

Some MKV files (but not all) containing ASS subtitles contain a nul character at the end of the ASS header (this is text). I do not know if this character is valid or not.

When the subtitle track is extracted using ffmpeg, the nul character can be seen in the middle of the ASS text file, at the end of the "[V4+ Styles]" section, before the "[Events]" section. Then, when this file is used to burn the subtitles using the "ass" video filter, no subtitle is seen in the video. If we manually modify the ASS text file to remove the nul character and retry the "ass" video filter, then the subtitles are correctly inserted in the video.

Even if the nul character is invalid (to be confirmed), failing because of it is a pity. Other tools handle that situation without problem:

  1. VLC correctly displays the subtitle track when playing the MKV file.
  2. When mkvextract is used to extract the subtitle track, the created ASS text file does not have the nul character.

So, for the sake of robustness, whether the nul character in the input file is valid or not, I suggest the following two enhancements. They are redundant in the specific example of this report, but they are in fact two independent robustness improvements for distinct use cases.

  1. When extracting the ASS header in a text file, remove the nul character, just like mkvextract.
  2. In the "ass" video filter, ignore nul characters from the text file containing the subtitles.

Demonstration

I have a small MKV file named im.mkv, 10 MB, 30 seconds of playback, demonstrating the problem. This file is extracted from a larger MKV file. Several other MKV from multiple origins have the same nul character in the ASS header.

Extracting the subtitles using mkvextract into a text file named im-mkvextract.ass:

$ mkvextract tracks im.mkv 2:im-mkvextract.ass
Extracting track 2 with the CodecID 'S_TEXT/ASS' to the file 'im-mkvextract.ass'. Container format: SSA/ASS text subtitles
Progress: 100%
$

Extracting the subtitles using ffmpeg into a text file named im-ffmpeg.ass:

$ ffmpeg -i im.mkv -vn -an -codec:s ass -f ass -y im-ffmpeg.ass
ffmpeg version N-81256-gd3426fb Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-libebur128 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
  libavutil      55. 28.100 / 55. 28.100
  libavcodec     57. 51.100 / 57. 51.100
  libavformat    57. 44.100 / 57. 44.100
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 49.100 /  6. 49.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
Input #0, matroska,webm, from 'im.mkv':
  Metadata:
    ENCODER         : Lavf57.25.100
  Duration: 00:00:30.60, start: 0.000000, bitrate: 2618 kb/s
    Chapter #0:0: start 0.000000, end 30.000000
    Metadata:
      title           : 00:00:00.000
    Stream #0:0(eng): Audio: ac3, 48000 Hz, 5.1(side), fltp, 384 kb/s
    Metadata:
      title           : Surround
      DURATION        : 00:00:30.016000000
    Stream #0:1: Video: h264 (High), yuv420p(tv, bt709), 1920x800 [SAR 1:1 DAR 12:5], 23.98 fps, 23.98 tbr, 1k tbn, 180k tbc (default)
    Metadata:
      DURATION        : 00:00:30.182000000
    Stream #0:2(fre): Subtitle: ass
    Metadata:
      DURATION        : 00:00:30.601000000
[ass @ 0000000002865500] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
Output #0, ass, to 'im-ffmpeg.ass':
  Metadata:
    encoder         : Lavf57.44.100
    Chapter #0:0: start 0.000000, end 30.000000
    Metadata:
      title           : 00:00:00.000
    Stream #0:0(fre): Subtitle: ass
    Metadata:
      DURATION        : 00:00:30.601000000
      encoder         : Lavc57.51.100 ass
Stream mapping:
  Stream #0:2 -> #0:0 (ass (ssa) -> ass (native))
Press [q] to stop, [?] for help
size=       1kB time=00:00:28.68 bitrate=   0.4kbits/s speed=50.9x
video:0kB audio:0kB subtitle:1kB other streams:0kB global headers:0kB muxing overhead: 167.719299%
$

The file im-ffmpeg.ass contains a nul character as illustrated below. The file im-mkvextract.ass does not have this nul character.

[Script Info]
ScriptType: v4.00+
Collisions: Normal
PlayResX: 1920
PlayResY: 800
Timer: 100.0
WrapStyle: 0

[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Default,Arial,52,&H00FFFFFF,&H00FFFFFF,&H000F0F0F,&H000F0F0F,0,0,0,0,100,100,0,0.00,1,2,3,2,20,20,20,0
^@ <==== nul character here
[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
.....

Now, we try to burn the two subtitle files im-ffmpeg.ass and im-mkvextract.ass in the video, separately.

In the file im1.mpg, generated using im-ffmpeg.ass, there is no subtitle in the video. In the file im2.mpg, generated using im-mkvextract.ass, the subtitles are correctly inserted in the video.

$ ffmpeg -i im.mkv -codec:a ac3 -codec:v mpeg2video -vf ass=filename=im-ffmpeg.ass -f dvd -y im1.mpg
ffmpeg version N-81256-gd3426fb Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-libebur128 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
  libavutil      55. 28.100 / 55. 28.100
  libavcodec     57. 51.100 / 57. 51.100
  libavformat    57. 44.100 / 57. 44.100
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 49.100 /  6. 49.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
Input #0, matroska,webm, from 'im.mkv':
  Metadata:
    ENCODER         : Lavf57.25.100
  Duration: 00:00:30.60, start: 0.000000, bitrate: 2618 kb/s
    Chapter #0:0: start 0.000000, end 30.000000
    Metadata:
      title           : 00:00:00.000
    Stream #0:0(eng): Audio: ac3, 48000 Hz, 5.1(side), fltp, 384 kb/s
    Metadata:
      title           : Surround
      DURATION        : 00:00:30.016000000
    Stream #0:1: Video: h264 (High), yuv420p(tv, bt709), 1920x800 [SAR 1:1 DAR 12:5], 23.98 fps, 23.98 tbr, 1k tbn, 180k tbc (default)
    Metadata:
      DURATION        : 00:00:30.182000000
    Stream #0:2(fre): Subtitle: ass
    Metadata:
      DURATION        : 00:00:30.601000000
[Parsed_ass_0 @ 000000000073ff20] Shaper: FriBidi 0.19.6 (SIMPLE)
[Parsed_ass_0 @ 000000000073ff20] Using font provider directwrite
[Parsed_ass_0 @ 000000000073ff20] Added subtitle file: 'im-ffmpeg.ass' (2 styles, 0 events)
[dvd @ 00000000027f8500] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
    Last message repeated 1 times
[dvd @ 00000000027f8500] VBV buffer size not set, using default size of 130KB
If you want the mpeg file to be compliant to some specification
Like DVD, VCD or others, make sure you set the correct buffer size
Output #0, dvd, to 'im1.mpg':
  Metadata:
    encoder         : Lavf57.44.100
    Chapter #0:0: start 0.000000, end 30.000000
    Metadata:
      title           : 00:00:00.000
    Stream #0:0: Video: mpeg2video (Main), yuv420p, 1920x800 [SAR 1:1 DAR 12:5], q=2-31, 200 kb/s, 23.98 fps, 90k tbn, 23.98 tbc (default)
    Metadata:
      DURATION        : 00:00:30.182000000
      encoder         : Lavc57.51.100 mpeg2video
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
    Stream #0:1(eng): Audio: ac3, 48000 Hz, 5.1(side), fltp, 448 kb/s
    Metadata:
      title           : Surround
      DURATION        : 00:00:30.016000000
      encoder         : Lavc57.51.100 ac3
Stream mapping:
  Stream #0:1 -> #0:0 (h264 (native) -> mpeg2video (native))
  Stream #0:0 -> #0:1 (ac3 (native) -> ac3 (native))
Press [q] to stop, [?] for help
frame=  724 fps=126 q=24.8 Lsize=    6742kB time=00:00:30.11 bitrate=1834.1kbits/s dup=22 drop=0 speed=5.23x
video:4832kB audio:1642kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 4.153100%
$
$
$ ffmpeg -i im.mkv -codec:a ac3 -codec:v mpeg2video -vf ass=filename=im-mkvextract.ass -f dvd -y im2.mpg
ffmpeg version N-81256-gd3426fb Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-libebur128 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
  libavutil      55. 28.100 / 55. 28.100
  libavcodec     57. 51.100 / 57. 51.100
  libavformat    57. 44.100 / 57. 44.100
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 49.100 /  6. 49.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
Input #0, matroska,webm, from 'im.mkv':
  Metadata:
    ENCODER         : Lavf57.25.100
  Duration: 00:00:30.60, start: 0.000000, bitrate: 2618 kb/s
    Chapter #0:0: start 0.000000, end 30.000000
    Metadata:
      title           : 00:00:00.000
    Stream #0:0(eng): Audio: ac3, 48000 Hz, 5.1(side), fltp, 384 kb/s
    Metadata:
      title           : Surround
      DURATION        : 00:00:30.016000000
    Stream #0:1: Video: h264 (High), yuv420p(tv, bt709), 1920x800 [SAR 1:1 DAR 12:5], 23.98 fps, 23.98 tbr, 1k tbn, 180k tbc (default)
    Metadata:
      DURATION        : 00:00:30.182000000
    Stream #0:2(fre): Subtitle: ass
    Metadata:
      DURATION        : 00:00:30.601000000
[Parsed_ass_0 @ 00000000027a96c0] Shaper: FriBidi 0.19.6 (SIMPLE)
[Parsed_ass_0 @ 00000000027a96c0] Using font provider directwrite
[Parsed_ass_0 @ 00000000027a96c0] Added subtitle file: 'im-mkvextract.ass' (2 styles, 12 events)
[dvd @ 00000000029700a0] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
    Last message repeated 1 times
[dvd @ 00000000029700a0] VBV buffer size not set, using default size of 130KB
If you want the mpeg file to be compliant to some specification
Like DVD, VCD or others, make sure you set the correct buffer size
Output #0, dvd, to 'im2.mpg':
  Metadata:
    encoder         : Lavf57.44.100
    Chapter #0:0: start 0.000000, end 30.000000
    Metadata:
      title           : 00:00:00.000
    Stream #0:0: Video: mpeg2video (Main), yuv420p, 1920x800 [SAR 1:1 DAR 12:5], q=2-31, 200 kb/s, 23.98 fps, 90k tbn, 23.98 tbc (default)
    Metadata:
      DURATION        : 00:00:30.182000000
      encoder         : Lavc57.51.100 mpeg2video
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
    Stream #0:1(eng): Audio: ac3, 48000 Hz, 5.1(side), fltp, 448 kb/s
    Metadata:
      title           : Surround
      DURATION        : 00:00:30.016000000
      encoder         : Lavc57.51.100 ac3
Stream mapping:
  Stream #0:1 -> #0:0 (h264 (native) -> mpeg2video (native))
  Stream #0:0 -> #0:1 (ac3 (native) -> ac3 (native))
Press [q] to stop, [?] for help
[Parsed_ass_0 @ 00000000027a96c0] fontselect: (Arial, 400, 0) -> ArialMT, 0, ArialMTp=20 drop=0 speed=5.77x
frame=  724 fps=110 q=24.8 Lsize=    6914kB time=00:00:30.11 bitrate=1880.9kbits/s dup=22 drop=0 speed= 4.6x
video:5003kB audio:1642kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 4.057209%
$

Attachments (1)

im-short.mkv​ (2.4 MB ) - added by Thierry Lelegard 8 years ago.
Shorter version of im.mkv to fit within upload limits (same issue but only 2 subtitle frames))

Change History (3)

by Thierry Lelegard, 8 years ago

Attachment: im-short.mkv​ added

Shorter version of im.mkv to fit within upload limits (same issue but only 2 subtitle frames))

comment:1 by Carl Eugen Hoyos, 8 years ago

Component: ffmpeg → undetermined
Keywords: ass added

comment:2 by Carl Eugen Hoyos, 6 years ago

Keywords: mkv added
Reproduced by developer: set
Status: new → open
Type: enhancement → defect
Note: See TracTickets for help on using tickets.