Opened 3 years ago

Closed 2 years ago

#2024 closed defect (fixed)

Reencoding dvbsub fails

Reported by: cehoyos Owned by: er.anshul.maheshwari@gmail.com
Priority: normal Component: avcodec
Version: git-master Keywords: dvbsub av_interleaved_write_frame
Cc: er.anshul.maheshwari@gmail.com Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: yes

Description

I tested two samples:
http://samples.ffmpeg.org/MPEG-VOB/ClosedCaptions/Starship_Troopers.vob
http://samples.ffmpeg.org/MPEG-VOB/transport/dvbsub/dvbsubtest.ts
and tried to reencode them to dvbsub. This works fine for the dvdsub sample, but fails for the dvbsub (input) sample.

$ ffmpeg -i dvbsubtest.ts -qscale 2 -mbd 2 -scodec dvbsub out.ts
ffmpeg version N-47774-g26db653 Copyright (c) 2000-2012 the FFmpeg developers
  built on Dec 12 2012 21:57:33 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --disable-indev=jack
  libavutil      52. 12.100 / 52. 12.100
  libavcodec     54. 79.102 / 54. 79.102
  libavformat    54. 49.101 / 54. 49.101
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 26.102 /  3. 26.102
  libswscale      2.  1.103 /  2.  1.103
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[mpegts @ 0x2f23280] max_analyze_duration 5000000 reached at 5016000
[mpegts @ 0x2f23280] PES packet size mismatch
Input #0, mpegts, from 'dvbsubtest.ts':
  Duration: 00:00:34.60, start: 79677.098467, bitrate: 4848 kb/s
  Program 1
    Stream #0:0[0x1901](eng): Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x19a1](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 256 kb/s
    Stream #0:2[0x19b1](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Please use -q:a or -q:v, -qscale is ambiguous
Output #0, mpegts, to 'out.ts':
  Metadata:
    encoder         : Lavf54.49.101
    Stream #0:0(eng): Video: mpeg2video (hq), yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 200 kb/s, 90k tbn, 25 tbc
    Stream #0:1(eng): Audio: mp2, 48000 Hz, stereo, s16, 128 kb/s
    Stream #0:2(eng): Subtitle: dvb_subtitle
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video -> mpeg2video)
  Stream #0:1 -> #0:1 (mp2 -> mp2)
  Stream #0:2 -> #0:2 (dvbsub -> dvbsub)
Press [q] to stop, [?] for help
frame=  310 fps=0.0 q=2.0 size=       0kB time=00:00:12.32 bitrate=   0.0kbits/s dup=10 drop=0Application provided invalid, non monotonically increasing dts to muxer in stream 2: 3005830 >= 1670230
av_interleaved_write_frame(): Invalid argument

Remuxing all strams fails (remuxing only audio and video works fine):

$ ffmpeg -i dvbsubtest.ts -acodec copy -vcodec copy -scodec dvbsub out.ts
ffmpeg version N-47774-g26db653 Copyright (c) 2000-2012 the FFmpeg developers
  built on Dec 12 2012 21:57:33 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --disable-indev=jack
  libavutil      52. 12.100 / 52. 12.100
  libavcodec     54. 79.102 / 54. 79.102
  libavformat    54. 49.101 / 54. 49.101
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 26.102 /  3. 26.102
  libswscale      2.  1.103 /  2.  1.103
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[mpegts @ 0x1b92280] max_analyze_duration 5000000 reached at 5016000
[mpegts @ 0x1b92280] PES packet size mismatch
Input #0, mpegts, from 'dvbsubtest.ts':
  Duration: 00:00:34.60, start: 79677.098467, bitrate: 4848 kb/s
  Program 1
    Stream #0:0[0x1901](eng): Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x19a1](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 256 kb/s
    Stream #0:2[0x19b1](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Output #0, mpegts, to 'out.ts':
  Metadata:
    encoder         : Lavf54.49.101
    Stream #0:0(eng): Video: mpeg2video ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 25 fps, 90k tbn, 25 tbc
    Stream #0:1(eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, 256 kb/s
    Stream #0:2(eng): Subtitle: dvb_subtitle
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
  Stream #0:2 -> #0:2 (dvbsub -> dvbsub)
Press [q] to stop, [?] for help
[mpegts @ 0x1b9a2e0] Application provided invalid, non monotonically increasing dts to muxer in stream 2: 3005830 >= 1670230
av_interleaved_write_frame(): Invalid argument

Remuxing only the subtitles work, but the resulting file shows no subtitles:

$ ffmpeg -i dvbsubtest.ts -qscale 2 -mbd 2 -scodec copy out.ts
ffmpeg version N-47774-g26db653 Copyright (c) 2000-2012 the FFmpeg developers
  built on Dec 12 2012 21:57:33 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --disable-indev=jack
  libavutil      52. 12.100 / 52. 12.100
  libavcodec     54. 79.102 / 54. 79.102
  libavformat    54. 49.101 / 54. 49.101
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 26.102 /  3. 26.102
  libswscale      2.  1.103 /  2.  1.103
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[mpegts @ 0x2ae8280] max_analyze_duration 5000000 reached at 5016000
[mpegts @ 0x2ae8280] PES packet size mismatch
Input #0, mpegts, from 'dvbsubtest.ts':
  Duration: 00:00:34.60, start: 79677.098467, bitrate: 4848 kb/s
  Program 1
    Stream #0:0[0x1901](eng): Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x19a1](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 256 kb/s
    Stream #0:2[0x19b1](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Please use -q:a or -q:v, -qscale is ambiguous
Output #0, mpegts, to 'out.ts':
  Metadata:
    encoder         : Lavf54.49.101
    Stream #0:0(eng): Video: mpeg2video (hq), yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 200 kb/s, 90k tbn, 25 tbc
    Stream #0:1(eng): Audio: mp2, 48000 Hz, stereo, s16, 128 kb/s
    Stream #0:2(eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video -> mpeg2video)
  Stream #0:1 -> #0:1 (mp2 -> mp2)
  Stream #0:2 -> #0:2 (copy)
Press [q] to stop, [?] for help
frame=  311 fps=0.0 q=2.0 size=    5961kB time=00:00:13.09 bitrate=3728.0kbits/s dup=10 drop=0frame=  570 fps=569 q=2.0 size=   14614kB time=00:00:23.13 bitrate=5173.9kbits/s dup=10 drop=0frame=  821 fps=546 q=2.0 size=   25327kB time=00:00:32.76 bitrate=6333.3kbits/s dup=10 drop=0PES packet size mismatch
[mpeg2video @ 0x2aec140] invalid mb type in P Frame at 43 16
[mpeg2video @ 0x2aec140] Warning MVs not available
[mpeg2video @ 0x2aec140] concealing 900 DC, 900 AC, 900 MV errors in P frame
[mp2 @ 0x2aecc20] incomplete frame
Error while decoding stream #0:1: Invalid data found when processing input
frame=  866 fps=538 q=2.0 Lsize=   28381kB time=00:00:34.67 bitrate=6704.4kbits/s dup=12 drop=0
video:25613kB audio:535kB subtitle:50 global headers:0kB muxing overhead 8.332572%

Attachments (1)

ffmpeg_dvbsubs_queue_clear.patch (3.0 KB) - added by wvdschel 3 years ago.
A patch that adds queueing to OutputStream? to sidestep this bug.

Download all attachments as: .zip

Change History (16)

comment:1 Changed 3 years ago by Cigaes

For the dvbdub → dvbsub case, the input sample is strange, at it has two consecutive subtitles packets with data in them. Normally, dvbsub comes with a packet of data, then an empty packet to clear it.

ffmpeg is still not completely prepared to work with the various kind of subtitles packets, and it is a tricky API design question.

comment:2 Changed 3 years ago by cehoyos

Fails now slightly differently:

$ ffmpeg -i dvbsubtest.ts -qscale 2 -mbd 2 -scodec dvbsub out.ts
ffmpeg version N-48478-g6a9af92 Copyright (c) 2000-2013 the FFmpeg developers
  built on Jan  5 2013 14:56:18 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --disable-indev=jack
  libavutil      52. 13.100 / 52. 13.100
  libavcodec     54. 85.101 / 54. 85.101
  libavformat    54. 59.103 / 54. 59.103
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 32.100 /  3. 32.100
  libswscale      2.  1.103 /  2.  1.103
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[mpegts @ 0x226e880] max_analyze_duration 5000000 reached at 5016000
[mpegts @ 0x226e880] PES packet size mismatch
Input #0, mpegts, from 'dvbsubtest.ts':
  Duration: 00:00:34.60, start: 79677.098467, bitrate: 4848 kb/s
  Program 1
    Stream #0:0[0x1901](eng): Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x19a1](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 256 kb/s
    Stream #0:2[0x19b1](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Please use -q:a or -q:v, -qscale is ambiguous
Output #0, mpegts, to 'out.ts':
  Metadata:
    encoder         : Lavf54.59.103
    Stream #0:0(eng): Video: mpeg2video (hq), yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 200 kb/s, 90k tbn, 25 tbc
    Stream #0:1(eng): Audio: mp2, 48000 Hz, stereo, s16, 128 kb/s
    Stream #0:2(eng): Subtitle: dvb_subtitle
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video -> mpeg2video)
  Stream #0:1 -> #0:1 (mp2 -> mp2)
  Stream #0:2 -> #0:2 (dvbsub -> dvbsub)
Press [q] to stop, [?] for help
frame=  277 fps=0.0 q=2.0 size=       0kB time=00:00:11.00 bitrate=   0.0kbits/s dup=10 drop=0Application provided invalid, non monotonically increasing dts to muxer in stream 2: 3005830 >= 1670230
av_interleaved_write_frame(): Invalid argument
Version 0, edited 3 years ago by cehoyos (next)

comment:3 Changed 3 years ago by ubitux

Another sample with a similar issue in #2989. Probably affects any DVB sub sample.

comment:4 Changed 3 years ago by wvdschel

I've worked out what the problem is (with the help of nevcariel, ubitux and others on #ffmpeg-devel):

  • DVB subtitles are encoded as a series of rectangular bitmaps, and they are individually "flushed" to be displayed using a display segment packet.
  • The way the DVB subtitles decoder (avcodec/dvbsubdec.c) currently works, is that it keeps track of all the other segments (which define regions, colour lookup tables, and contain the actual bitmaps) until a display segment is encountered, at which point it renders all these things to an AVSubtitle.
  • On the encoding side, DVB subtitles differ from other subtitle codecs because it is currently the only codec that needs to have two packets to do its work: one to render the subtitle and one to indicate that it can be removed from the screen. These have (obviously) different PTS'es.
  • It's possible that a later AVSubtitle appears before an earlier one dissappears from the screen. In fact, it's quite common for live subtitling, where each word is pushed & displayed as a separate region (have a look at http://samples.ffmpeg.org/MPEG-VOB/transport/dvbsub/dvbsubtest.ts for an example of this happening).
  • Because the current subtitle encoding happens to write out the display & clear packets for DVB in sequence, a later subtitle can trigger the above error in the muxer.

So this is the problem. As for the solution: I don't have a clue on how to fix this cleanly. There's already quite a bit of DVB subtitles specific code in ffmpeg.c, and I'd rather remove from it than add to it, fixing the issue in the decoder.

I'd love to get this fixed, so I'm hoping that someone has an idea how this could be solved.

comment:5 Changed 3 years ago by wvdschel

Hi,

I modified ffmpeg to queue the clear packet of DVB subtitles instead of writing it out. I'm not sure if this is acceptable, and I'm pretty sure the code is not upto the required standards.

I'd like some feedback on wether this is an acceptable solution or what I have to do to get this merged.

Changed 3 years ago by wvdschel

A patch that adds queueing to OutputStream? to sidestep this bug.

comment:6 follow-up: Changed 3 years ago by cehoyos

Please send patches to the ffmpeg-devel mailing list where they get more attention.

comment:7 in reply to: ↑ 6 Changed 3 years ago by wvdschel

Replying to cehoyos:

Please send patches to the ffmpeg-devel mailing list where they get more attention.

I have, I just used the wrong email, so it's awaiting moderation.

comment:8 Changed 2 years ago by joolzg

Can I add something in so people will understand how DVBSUBS work.

Dvb subs works on an idea of regions and objects, there are NO CLEAR SUBS commands, what is sent is a new region list with 0 regions, there should be no code to explicitly send clear screen or anything like this.

What will happen is this

New region with 2 objects
New region with 1 object
New region with 0 objects
New region with 1 object

Now all these new regions have a "timeout" which says that if NO NEW DATA arrives then clear the screen as sonething has gone wrong, stream corrupted, out of sync etc.

The current ffmpeg fails in this as it does as posted above and tries to create a "clear screen" packet when it SHOULD NOT.

What should happen is that once encoding it should check the timeout perion and if expired send a "New region with 0 objects", this means we can remove the hack in ffmpeg which makes up 2 packets when it should not.

comment:9 Changed 2 years ago by mrlika

"Remuxing only the subtitles work, but the resulting file shows no subtitles" - this issue from head should be fixed with https://github.com/FFmpeg/FFmpeg/commit/c917cde9cc52ad1ca89926a617f847bc9861d5a0 commit.

comment:10 Changed 2 years ago by cehoyos

Remuxing the dvb subtitles was fixed by Serhii Marchuk in c917cde9
The problems with transcoding dvb subtitles are still reproducible.

comment:11 Changed 2 years ago by er.anshul.maheshwari@gmail.com

  • Owner set to er.anshul.maheshwari@gmail.com
  • Status changed from new to open

comment:12 Changed 2 years ago by er.anshul.maheshwari@gmail.com

  • Analyzed by developer set
  • Component changed from undetermined to avcodec
  • Reproduced by developer set

comment:13 Changed 2 years ago by er.anshul.maheshwari@gmail.com

Problem is due to setting end_display_time using page_time_out in dvbsub decoder, where spec specify decoder should not rely only on page_time_out.

comment:14 Changed 2 years ago by er.anshul.maheshwari@gmail.com

  • Cc er.anshul.maheshwari@gmail.com added

comment:15 Changed 2 years ago by cehoyos

  • Resolution set to fixed
  • Status changed from open to closed

Fixed by Anshul in 7e6cf364

Note: See TracTickets for help on using tickets.