Opened 5 years ago

Closed 4 years ago

#598 closed defect (fixed)

Subtitle on some BD/m2ts files are not detected because CRC check fails

Reported by: Sebastiii Owned by: michael
Priority: normal Component: avformat
Version: git-master Keywords: pgssub mpegts pmt bluray crc
Cc: skank Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Hello,

On some BD/M2TS files, ffplay/ffmpeg didn't detect subtitle.
It seems related that Ffmpeg relys it on the pmt.

This is a sample for testing :
http://dl.dropbox.com/u/10536084/mp/Sample/00002_nosubs_detected.m2ts

It only detect this :

ffprobe version N-33818-gd049257, Copyright (c) 2007-2011 the FFmpeg developers
  built on Oct 19 2011 22:57:39 with gcc 4.6.1
  configuration: --enable-gpl --enable-version3 --enable-runtime-cpudetect --ena
ble-avisynth --enable-bzlib --enable-frei0r --enable-libopencore-amrnb --enable-
libopencore-amrwb --enable-libfreetype --enable-libgsm --enable-libmp3lame --ena
ble-libopenjpeg --enable-librtmp --enable-libschroedinger --enable-libspeex --en
able-libtheora --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis
--enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib

  libavutil    51. 22. 0 / 51. 22. 0
  libavcodec   53. 22. 0 / 53. 22. 0
  libavformat  53. 17. 0 / 53. 17. 0
  libavdevice  53.  4. 0 / 53.  4. 0
  libavfilter   2. 44. 1 /  2. 44. 1
  libswscale    2.  1. 0 /  2.  1. 0
  libpostproc  51.  2. 0 / 51.  2. 0
[mpegts @ 01B58BC0] PES packet size mismatch
[mpegts @ 01B58BC0] PES packet size mismatch
[mpegts @ 01B58BC0] PES packet size mismatch
[mpegts @ 01B58BC0] PES packet size mismatch
[mpegts @ 01B58BC0] PES packet size mismatch
[mpegts @ 01B58BC0] PES packet size mismatch
Input #0, mpegts, from 'E:\!BDSample\00002.m2ts':
  Duration: 00:01:05.53, start: 1.991000, bitrate: 32881 kb/s
  Program 1
  No Program
    Stream #0:0[0x1011]: Video: h264 (High), yuv420p, 1920x1080 [SAR 1:1 DAR 16:
9], 23.98 fps, 23.98 tbr, 90k tbn, 47.95 tbc
    Stream #0:1[0x1102]: Audio: ac3, 48000 Hz, 5.1(side), s16, 448 kb/s
    Stream #0:2[0x1103]: Audio: ac3, 48000 Hz, 5.1(side), s16, 448 kb/s
    Stream #0:3[0x1104]: Audio: ac3, 48000 Hz, 5.1(side), s16, 448 kb/s
    Stream #0:4[0x1105]: Audio: ac3, 48000 Hz, 5.1(side), s16, 448 kb/s
    Stream #0:5[0x1101]: Audio: ac3, 48000 Hz, stereo, s16, 224 kb/s
    Stream #0:6[0x1100]: Audio: dts (DTS-HD MA), 48000 Hz, 5.1(side), s16, 1536
kb/s

Thanks if you can fix it :)

Attachments (2)

patchpgs.diff (3.9 KB) - added by cehoyos 5 years ago.
patchpgs2.diff (3.9 KB) - added by cehoyos 4 years ago.

Download all attachments as: .zip

Change History (31)

comment:1 Changed 5 years ago by cehoyos

  • Component changed from FFmpeg to undetermined

Which application shows subtitles for the uploaded stream?
(How do you know that the stream contains subtitles?)

comment:2 Changed 5 years ago by Sebastiii

Hi cehoyos,

I use mpc-hc and Mediaportal with Lav Splitter based on ffmpeg.
I use ffplay from http://ffmpeg.zeranoe.com/builds/ and subs are not detected.
I have try xine on Ubuntu and didn't work.

If i use TsRemux?, i get this (5 Subtitles):

MUXOPT --no-pcr-on-video-pid --new-audio-pes --vbr  --vbv-len=500
V_MPEG4/ISO/AVC, "00002_nosubs_detected.m2ts", fps=23.976, insertSEI, contSPS, track=4113
A_DTS, "00002_nosubs_detected.m2ts", timeshift=-8ms, track=4352
A_AC3, "00002_nosubs_detected.m2ts", timeshift=-8ms, track=4353
A_AC3, "00002_nosubs_detected.m2ts", timeshift=-8ms, track=4354
A_AC3, "00002_nosubs_detected.m2ts", timeshift=-8ms, track=4355
A_AC3, "00002_nosubs_detected.m2ts", timeshift=-8ms, track=4356
A_AC3, "00002_nosubs_detected.m2ts", timeshift=-8ms, track=4357
S_HDMV/PGS, "00002_nosubs_detected.m2ts",bottom-offset=24,font-border=2,text-align=center,video-width=1920,video-height=1080,fps=23.976, track=4608
S_HDMV/PGS, "00002_nosubs_detected.m2ts",bottom-offset=24,font-border=2,text-align=center,video-width=1920,video-height=1080,fps=23.976, track=4609
S_HDMV/PGS, "00002_nosubs_detected.m2ts",bottom-offset=24,font-border=2,text-align=center,video-width=1920,video-height=1080,fps=23.976, track=4610
S_HDMV/PGS, "00002_nosubs_detected.m2ts",bottom-offset=24,font-border=2,text-align=center,video-width=1920,video-height=1080,fps=23.976, track=4611
S_HDMV/PGS, "00002_nosubs_detected.m2ts",bottom-offset=24,font-border=2,text-align=center,video-width=1920,video-height=1080,fps=23.976, track=4612

comment:3 Changed 5 years ago by reimar

MPlayer can show the subtitles (though you can select them only from the point where the first packets appear).

comment:4 Changed 5 years ago by Sebastiii

Thanks reimar,
I have run the file in ffplay and wait when first subtitle packet should comes (40s) and i didn't see it.

comment:5 Changed 5 years ago by cehoyos

  • Component changed from undetermined to avformat
  • Reproduced by developer set
  • Status changed from new to open

Press "j" to switch subtitles in MPlayer (once they are available).

comment:6 Changed 5 years ago by Sebastiii

Hi,

Thanks,
I think i have a similar issue with this sample, should i open another ticket ?
http://dl.dropbox.com/u/10536084/mp/Sample/Hop_no_subs_detected.vob
Seb.

Changed 5 years ago by cehoyos

comment:7 follow-ups: Changed 5 years ago by cehoyos

I see three different problems with the BD stream you provided:
First, subtitle streams (0x120x) start late and are not detected by ffmpeg -i. The same happens for the DVD stream. Since (subtitle) streams can start at any point while playing back, but not every user wants a whole file to be scanned when starting ffmpeg, I do not think there is an easy solution. Note that the DVD stream plays as expected with mplayer -demuxer lavf -sid 0 (and ffplay with "t").
At least ffmpeg -analyzeduration 2000000000 -i stream and ffplay stream -sst 0 should work, both currently fail.

The second problem is that for the given BD sample, no streams are detected by the MPEG-TS demuxer, all have to be auto-detected (note the missing codec_tag information for the found streams in the output of ffmpeg -i). Since mplayer -demuxer mpegts works fine (it does not support auto-detection of video streams except MPEG-2), there seems to be a bug in FFmpeg's demuxer (not a regression, afaict). If you cut the sample so the subtitle streams start near the beginning of the file, subtitles are only shown as "Data: [0][0][0][0] / 0x0000".
I worked around this problem with attached patch (autodetection of BD subtitles), it works fine with mplayer -demuxer lavf -sid 0, but probably has a significant performance impact (at the moment the stream starts).

The third problem is that the BD subtitles are only shown for one frame by ffplay if you use the patch. I suspect this will not hit you when using third party applications (it cannot be reproduced with MPlayer), but is still noteworthy.

comment:8 Changed 5 years ago by Sebastiii

Ok :) thanks to look into it.
I will try now the patch to test third party application.
Cheers,
Seb.

comment:9 in reply to: ↑ 7 ; follow-up: Changed 4 years ago by Andrea3000

Replying to cehoyos:

I see three different problems with the BD stream you provided:
First, subtitle streams (0x120x) start late and are not detected by ffmpeg -i. The same happens for the DVD stream. Since (subtitle) streams can start at any point while playing back, but not every user wants a whole file to be scanned when starting ffmpeg, I do not think there is an easy solution. Note that the DVD stream plays as expected with mplayer -demuxer lavf -sid 0 (and ffplay with "t").
At least ffmpeg -analyzeduration 2000000000 -i stream and ffplay stream -sst 0 should work, both currently fail.

The second problem is that for the given BD sample, no streams are detected by the MPEG-TS demuxer, all have to be auto-detected (note the missing codec_tag information for the found streams in the output of ffmpeg -i). Since mplayer -demuxer mpegts works fine (it does not support auto-detection of video streams except MPEG-2), there seems to be a bug in FFmpeg's demuxer (not a regression, afaict). If you cut the sample so the subtitle streams start near the beginning of the file, subtitles are only shown as "Data: [0][0][0][0] / 0x0000".
I worked around this problem with attached patch (autodetection of BD subtitles), it works fine with mplayer -demuxer lavf -sid 0, but probably has a significant performance impact (at the moment the stream starts).

The third problem is that the BD subtitles are only shown for one frame by ffplay if you use the patch. I suspect this will not hit you when using third party applications (it cannot be reproduced with MPlayer), but is still noteworthy.

I have the same issue with PGS subtitles not been detected but the provided patch didn't work.
Is there any following with this problem?

comment:10 in reply to: ↑ 9 Changed 4 years ago by cehoyos

Replying to Andrea3000:

I have the same issue with PGS subtitles not been detected but the provided patch didn't work.
Is there any following with this problem?

Please open a new ticket with a ffmpeg command line, complete, uncut console output and a sample.

Changed 4 years ago by cehoyos

comment:11 Changed 4 years ago by cehoyos

Both samples in http://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket598/ show subtitles with mplayer -demuxer lavf -sid 0 and the attached (new) patch, sample_pgs_sub.m2ts also works with ffplay and "t" (it does not show the "third" problem from comment:7).

comment:12 Changed 4 years ago by Sebastiii

Thanks for the new patch, i will try to compil it for LAV Splitter suite from Nevcairiel and see what happen :)

comment:13 Changed 4 years ago by heleppkes

Instead of going the tedious parsing/auto detection route, i sat down and tried to find the issue in avformats MPEG-TS parsing.

The root cause of the problem seems to be that the CRC in the PAT/PMT in those files is not correct. avformats mpeg-ts demuxer will just ignore any PAT/PMT which has a wrong CRC, while mplayers demuxer ignores the CRC check.

In any case, i disabled the CRC check in avformat, and suddenly it finds all the streams just fine.
The changes required are here:
http://git.1f0.de/gitweb?p=ffmpeg.git;a=commitdiff;h=3ece9af0ea7b069a19a1d9f7b781703bc1641186

I'm not sure how many bad things can happen when the CRC is wrong, or how important that check is for broadcast content... i'll leave the checks disabled and see what happens. Nothing beats empirical evidence. :p

comment:14 follow-up: Changed 4 years ago by cehoyos

The change in pat_cb() alone seems sufficient.

Do you know why ffplay shows the subtitles in 00002_nosubs_detected.m2ts only for one frame?

comment:15 follow-up: Changed 4 years ago by heleppkes

For this file, yes, the pat_cb change is enough because only the PMTs CRC is wrong, however i also have a file where the PATs CRC is wrong, which would require the second change in mpegts_read_header as well.

I have not worked with ffmpegs subtitle functionality yet, so i cannot say if its a ffplay failure or a general subtitling problem. Its on my list of things to implement "soon", though.

comment:16 Changed 4 years ago by Sebastiii

Thanks Guys and Nevcairiel for the latest fix :)
It works :)

comment:17 in reply to: ↑ 15 Changed 4 years ago by cehoyos

Replying to heleppkes:

For this file, yes, the pat_cb change is enough because only the PMTs CRC is wrong, however i also have a file where the PATs CRC is wrong, which would require the second change in mpegts_read_header as well.

Could you provide the sample?

comment:18 in reply to: ↑ 14 Changed 4 years ago by Andrea3000

Replying to cehoyos:

The change in pat_cb() alone seems sufficient.

Do you know why ffplay shows the subtitles in 00002_nosubs_detected.m2ts only for one frame?

I know it.

It's a behavior that I've encountered only with Avatar's subtitles.
After quite every correct subtitles there is a blank subtitle with a start time that is few milliseconds later than (or even equal to) the start time of the previous correct subtitles. Since blank subtitles are used to mark clearing of old subtitle before displaying the new one, the correct subtitles is cleared immediadetly after having been shown.
I can also affirm that this issue is related to the timing because after the blank subtitle with wrong start time is decoded, the next frame is a new correct subtitle to be displayed and not the blank subtitles which should clear the old one at the right time. It seems that the blank subtitle has a start time which is equal to the start time of the previous correct subtitle rather than having the start time equal to the end time of the sub (the correct end time obviously, not the 20s timeout which the actual decoder set).

comment:19 in reply to: ↑ 7 Changed 4 years ago by cehoyos

Replying to cehoyos:

I see three different problems with the BD stream you provided:
First, subtitle streams (0x120x) start late and are not detected by ffmpeg -i. The same happens for the DVD stream. Since (subtitle) streams can start at any point while playing back, but not every user wants a whole file to be scanned when starting ffmpeg, I do not think there is an easy solution. Note that the DVD stream plays as expected with mplayer -demuxer lavf -sid 0 (and ffplay with "t").
At least ffmpeg -analyzeduration 2000000000 -i stream and ffplay stream -sst 0 should work, both currently fail.

The subtitles in the dvd-stream can be detected with:
$ ffmpeg -probesize 60M -analyzeduration 60M -i Hop_no_subs_detected.vob
This seems mostly expected behaviour.
Without the CRC problem, something similar is possible with the Bluray stream.

comment:20 follow-up: Changed 4 years ago by skank

Im having issues with pgs subtitles in bdmv/m2ts movies too
When i try to play them, (with vlc or xbmc player) the subs disappear/vanish immediately..' so this is really a ffmpeg issue. When will this be fixed?
How do i apply this patch to see if it works?
Thx

comment:21 in reply to: ↑ 20 Changed 4 years ago by cehoyos

  • Cc skank added
  • Keywords mpegts added; m2ts removed

Replying to skank:

When will this be fixed?

Did you provide a sample?

comment:22 follow-up: Changed 4 years ago by Sebastiii

Hi,

Sorry i was remove the sample few week ago, didn't remember that it was linked to this issue.
I'm reuploading it and eta/finish upload should be in 1 hour.

Thanks.

comment:23 in reply to: ↑ 22 Changed 4 years ago by cehoyos

Replying to Sebastiii:

Sorry i was remove the sample few week ago, didn't remember that it was linked to this issue.
I'm reuploading it and eta/finish upload should be in 1 hour.

Your sample is still available, another user suggested that one part of the issue - that subtitles disappear immediately - is important so I asked if he can provide an additional sample.

comment:24 Changed 4 years ago by DoomBot

I too am having a problem with subtitles disappearing immediately just like skank said.

http://dl.dropbox.com/u/103661855/Sample...ssue-.m2ts

Version 0, edited 4 years ago by DoomBot (next)

comment:25 Changed 4 years ago by Sebastiii

I have tested the new sample (with LAV Splittter + Mediaportal) and subs are ok.
Lav Splitter has heleppkes fix.

comment:26 Changed 4 years ago by skank

Xbmc uses ffmpeg and not lav splitter.. Thatswhy we posted here.. Ffmpeg should have a fix for this...
Isnt it possible to fix for ffmpeg so xbmc is fixed too... And vlc too..

comment:27 Changed 4 years ago by cehoyos

  • Keywords pgssub added; subtitle removed
  • Summary changed from Subtitle on some BD/m2ts files are not handle. to Subtitle on some BD/m2ts files are not detected because CRC check fails

Ticket #1722 is about the disappearing of subtitles, please use this ticket only for the CRC problem when reading subtitles in transport streams. We currently have one sample in http://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket598/ - is there an additional sample?

comment:28 in reply to: ↑ 7 Changed 4 years ago by cehoyos

Replying to cehoyos:

The third problem is that the BD subtitles are only shown for one frame by ffplay if you use the patch. I suspect this will not hit you when using third party applications (it cannot be reproduced with MPlayer), but is still noteworthy.

This problem was fixed by Nicolas George, the only remaining problem is the CRC check in libavformat/mpegts.c, I cannot really comment on that issue.

comment:29 Changed 4 years ago by michael

  • Keywords crc added
  • Resolution set to fixed
  • Status changed from open to closed
Note: See TracTickets for help on using tickets.