Opened 5 months ago

Closed 8 weeks ago

#7506 closed defect (fixed)

EIA-608 Closed Captions distortion on the output.

Reported by: w_boba Owned by:
Priority: normal Component: ffmpeg
Version: git-master Keywords: cc fps
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:
How to reproduce:

% ffmpeg.exe -y -v 99 -loglevel 99 -i sample-1min.ts -c:a aac -c:v libx264 sample-1min.mp4
ffmpeg version N-92202-gf72b9904fe Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 8.2.1 (GCC) 20181017
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
  libavutil      56. 19.101 / 56. 19.101
  libavcodec     58. 33.101 / 58. 33.101
  libavformat    58. 19.100 / 58. 19.100
  libavdevice    58.  4.105 / 58.  4.105
  libavfilter     7. 33.101 /  7. 33.101
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100

When I try to re-encode some MPEG2 TS files into h.264/mp4 using libx264 for encoder, embedded closed captions are not properly passed through to output file. It looks to me like CCs are doubled for some reason.

Original CCs (got them with CCextractor tool) look like this:

1
00:00:01,751 --> 00:00:03,469
        Yup, so you just
       want a nice layer.

2
00:00:03,536 --> 00:00:04,670
       That's good there.

3
00:00:04,721 --> 00:00:06,305
      Potatoes are ready.

4
00:00:06,372 --> 00:00:08,090
  This gets covered in
 your famous pot crack?

While extracted from MP4 file after re-encoding - they look like this:

1
00:00:01,234 --> 00:00:03,034
   So we'e're makaking B.C. h h?

2
00:00:03,036 --> 00:00:04,703
        YuYup, so you u just
       want a  nice layerer.

3
00:00:04,871 --> 00:00:06,004
       That''s goodod there.

4
00:00:06,105 --> 00:00:07,672
      Potatoes a are ready..

Attachments (4)

sample-1min-stderr.log (411.4 KB) - added by w_boba 5 months ago.
FFmpeg debug log
sample-1min-ts.srt (2.9 KB) - added by w_boba 5 months ago.
CCs from original file
sample-1min-mp4.srt (2.9 KB) - added by w_boba 5 months ago.
CCs from output file
sample-1min-patched.srt (2.9 KB) - added by w_boba 4 months ago.
SRT file extracted from output MP4 file after patch was applied

Download all attachments as: .zip

Change History (15)

Changed 5 months ago by w_boba

FFmpeg debug log

Changed 5 months ago by w_boba

CCs from original file

Changed 5 months ago by w_boba

CCs from output file

comment:1 Changed 5 months ago by w_boba

All files required to reproduce this issue can be downloaded from here:
(240MB) http://64.137.230.232/cc-distortion.zip

comment:2 Changed 5 months ago by The_Bart_The

I recently found out FFMPEG preserves closed caption data by default and am currently in the process of switching my scripts over from HandBrakeCLI. I also discovered this, searched for this bug online and found this post the same day it went up.

Anyway, I'm getting this same bug when I convert an NBC broadcast (captured via antenna, so ATSC 1080i MPEG-2 TS; specifically from WMAQ in Chicago) but I haven't seen this bug yet from CBS (WBBM, 1080i), FOX (WFLD, 720p) PBS (WTTW, 1080i), CW (WPWR, 720p) or ABC (WLS, 720p).

I've only been using FFMPEG seriously for a few days, so I'll report back with any new relevant information. (Maybe it's only certain shows instead of networks; who knows?)

comment:3 Changed 4 months ago by The_Bart_The

I've done some research and I think I've cracked it.

Each frame of video can hold up to two characters of closed caption data. In line 5921 of your log, you'll see 367 duplicated frames and zero dropped frames. That's why you see duplicated letters.

If you use -vsync 0 like so:

% ffmpeg.exe -y -v 99 -loglevel 99 -i sample-1min.ts -c:a aac -c:v libx264 sample-1min.mp4 -vsync 0

FFmpeg won't try to adjust the framerate of your video by duplicating & dropping frames and your subtitles should remain intact.

I figured I was on to something when I remembered my local NBC affiliate was causing me the same problem. I remembered that all those years I was using HandBrakeCLI, NBC was the only network doing this to me:

https://i.imgur.com/vtOzpqM.png

Once I realized NBC was the only network changing framerates back and forth (Video and Film are code for 30 fps and 24 fps, respectively), and saw duplicated frames in my own FFmpeg logs I went looking for how to preserve the original framerate. I came across "-vsync 0" and here we are.

Of course, the bug's still valid despite this workaround.

comment:4 follow-up: Changed 4 months ago by cehoyos

Please test this patch:

diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index a12208c..c7e0806 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -1286,6 +1286,8 @@ static void do_video_out(OutputFile *of,
         ret = avcodec_send_frame(enc, in_picture);
         if (ret < 0)
             goto error;
+        // Make sure the Closed Captions will not be duplicated
+        av_frame_remove_side_data(in_picture, AV_FRAME_DATA_A53_CC);

         while (1) {
             ret = avcodec_receive_packet(enc, &pkt);

comment:5 in reply to: ↑ 4 Changed 4 months ago by w_boba

Replying to cehoyos:

Please test this patch:
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c

Thank you very much!

Looks like this patch works just as expected. I will do some more testing, but on the first try I've got this:

1
00:00:01,234 --> 00:00:03,001
   So we're making B.C. hash?   

2
00:00:03,003 --> 00:00:04,703
        Yup, so you just        
       want a nice layer.       

3
00:00:04,838 --> 00:00:05,970
       That's good there.       

4
00:00:06,072 --> 00:00:07,639
      Potatoes are ready.       

Comparing to "old" version:

1
00:00:01,234 --> 00:00:03,034
   So we'e're makaking B.C. h h?

2
00:00:03,036 --> 00:00:04,703
        YuYup, so you u just
       want a  nice layerer.

3
00:00:04,871 --> 00:00:06,004
       That''s goodod there.

4
00:00:06,105 --> 00:00:07,672
      Potatoes a are ready..

Changed 4 months ago by w_boba

SRT file extracted from output MP4 file after patch was applied

comment:6 Changed 4 months ago by w_boba

This patch works, when FPS filter is not being applied. With the following command problem persists.

I used same set of files from before for testing: ​http://64.137.230.232/cc-distortion.zip

See code blocks:

ffmpeg version N-92494-ge3a9630982 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)
ffmpeg -threads 0 -y -i sample-1min-ts.ts -c:a aac -c:v libx264 -filter:v yadif,fps=ntsc sample-1min-patched.mp4
1
00:00:01,234 --> 00:00:03,001
   So w we're m making B.C.C. h?

2
00:00:03,003 --> 00:00:04,769
        Yup, so yoyou just      
       want a a nice layayer.   

3
00:00:04,871 --> 00:00:06,137
       Thatat's gogood there..  

4
00:00:06,239 --> 00:00:07,639
      Potatoeses are readydy.   
Last edited 4 months ago by w_boba (previous) (diff)

comment:7 follow-up: Changed 4 months ago by cehoyos

  • Component changed from undetermined to ffmpeg
  • Keywords cc fps added; EIA-608 a53cc closed captions removed

Please also test this patch:

diff --git a/libavfilter/vf_fps.c b/libavfilter/vf_fps.c
index 9167a00..33a6079 100644
--- a/libavfilter/vf_fps.c
+++ b/libavfilter/vf_fps.c
@@ -249,6 +249,7 @@ static int write_frame(AVFilterContext *ctx, FPSContext *s, AVFilterLink *outlin
         frame = av_frame_clone(s->frames[0]);
         if (!frame)
             return AVERROR(ENOMEM);
+        av_frame_remove_side_data(s->frames[0], AV_FRAME_DATA_A53_CC);
         frame->pts = s->next_pts++;

         av_log(ctx, AV_LOG_DEBUG, "Writing frame with pts %"PRId64" to pts %"PRId64"\n",

comment:8 in reply to: ↑ 7 Changed 4 months ago by w_boba

Replying to cehoyos:

Please also test this patch:
diff --git a/libavfilter/vf_fps.c b/libavfilter/vf_fps.c
index 9167a00..33a6079 100644
--- a/libavfilter/vf_fps.c
+++ b/libavfilter/vf_fps.c

This one worked great! Thank you very much!

comment:9 Changed 4 months ago by cehoyos

  • Resolution set to fixed
  • Status changed from new to closed

comment:10 Changed 8 weeks ago by w_boba

  • Resolution fixed deleted
  • Status changed from closed to reopened

Exactly same problem with closed captions when using deinterlacing with frame doubler:

% ffmpeg.exe -y -v 99 -loglevel 99 -i sample-1min.ts -c:a aac -vf yadif=1:-1:0 -c:v libx264 sample-1min.mp4

Same CC data is attached to each frame produced from 2 fields.

comment:11 Changed 8 weeks ago by cehoyos

  • Resolution set to fixed
  • Status changed from reopened to closed

Please open a new ticket, provide the command line you tested including the complete, uncut console output. (Sorry that I didn't tell you originally.)

Note: See TracTickets for help on using tickets.