cc_dec decoder very slow at extracting CEA-708 closed captions from mpeg2ts file

[devesh]

Summary of the bug: When transcoding an mpeg2ts hdhomerun recording, if I ask ffmpeg to also decode the embedded closed captions, processing slows down much more than expected.
How to reproduce:

% time ~/projects/ffmpeg/ffmpeg -f lavfi -i "movie=1441_20180209070000.ts[out0+subcc]" out2.srt
ffmpeg version N-89982-g81d6501be7 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.2.0-8ubuntu3)                                      
  configuration: --enable-libmp3lame --enable-libfdk-aac --enable-cuda --enable-cuvid --enable-nvenc --enable-nonfree --enable-libnpp --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64                          
  libavutil      56.  7.100 / 56.  7.100                                        
  libavcodec     58. 10.100 / 58. 10.100                                        
  libavformat    58.  9.100 / 58.  9.100                                        
  libavdevice    58.  1.100 / 58.  1.100                                        
  libavfilter     7. 11.101 /  7. 11.101                                        
  libswscale      5.  0.101 /  5.  0.101                                        
  libswresample   3.  0.101 /  3.  0.101                                        
[mpegts @ 0x5590806ca500] PES packet size mismatch                              
Input #0, lavfi, from 'movie=1441_20180209070000.ts[out0+subcc]':               
  Duration: N/A, start: 93886.435011, bitrate: N/A                              
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 90k tbc                             
    Stream #0:1: Subtitle: eia_608      
Output #0, srt, to 'out2.srt':          
  Metadata:         
    encoder         : Lavf58.9.100      
    Stream #0:0: Subtitle: subrip (srt) 
    Metadata:       
      encoder         : Lavc58.10.100 srt                                       
Stream mapping:     
  Stream #0:1 -> #0:0 (eia_608 (cc_dec) -> subrip (srt))                        
Press [q] to stop, [?] for help         
[Parsed_movie_0 @ 0x5590806ca000] EOF timestamp not reliable=7.99x              
size=      46kB time=00:27:44.26 bitrate=   0.2kbits/s speed=7.99x              
video:0kB audio:0kB subtitle:30kB other streams:0kB global headers:0kB muxing overhead: 50.277382%  

real    3m28.511s   
user    3m21.523s   
sys     0m5.121s    

Compare to ccextractor on the same file:

% time ~/projects/ccextractor/linux/ccextractor 1441_20[21/1640$
000.ts -o ~/out.srt                                                             
CCExtractor 0.85, Carlos Fernandez Sanz, Volker Quetschke.                      
Teletext portions taken from Petr Kutalek's telxcc                              
--------------------------------------------------------------------------      
Input: 1441_20180209070000.ts                                                   
[Extract: 1] [Stream mode: Autodetect]                                          
[Program : Auto ] [Hauppage mode: No] [Use MythTV code: Auto]                   
[Timing mode: Auto] [Debug: No] [Buffer input: No]                              
[Use pic_order_cnt_lsb for H.264: No] [Print CC decoder traces: No]             
[Target format: .srt] [Encoding: UTF-8] [Delay: 0] [Trim lines: No]             
[Add font color data: Yes] [Add font typesetting: Yes]                          
[Convert case: No] [Video-edit join: No]                                        
[Extraction start time: not set (from start)]                                   
[Extraction end time: not set (to end)]                                         
[Live stream: No] [Clock frequency: 90000]                                      
[Teletext page: Autodetect]                                                     
[Start credits text: None]                                                      
                                                                                
-----------------------------------------------------------------               
Opening file: 1441_20180209070000.ts                                            
File seems to be a transport stream, enabling TS mode                           
Analyzing data in general mode          
Creating /home/devesh/out.srt           


New video information found             
[1920 * 1080] [AR: 03 - 16:9] [FR: 04 - 29.97] [progressive: no]                

  0%  |  00:00XDS Notice: Network call letters now KBCW                         
XDS Notice: Network is now CW           
100%  |  29:58      
Number of NAL_type_7: 0                 
Number of VCL_HRD: 0
Number of NAL HRD: 0
Number of jump-in-frames: 0             
Number of num_unexpected_sei_length: 0  

Total frames time:        00:29:59:030  (53917 frames at 29.97fps)

Min PTS:                                26:04:46:368
Max PTS:                                26:34:45:432
Length:                          00:29:59:064

Initial GOP time:          00:00:00:000
Final GOP time:          00:00:00:000+30F
Diff. GOP length:          00:00:00:000+30F     (00:00:01:001)


Total user data fields: 53917
HDTV type user data fields: 53917
Done, processing time = 14 seconds
Issues? Open a ticket here
https://github.com/CCExtractor/ccextractor/issues

real    0m13.910s
user    0m9.021s
sys     0m4.484s

[Carl Eugen Hoyos]

May be the same issue as #7542.

[Aman]

This is because the closed caption data is embedded inside the mpeg2 video frames, and ffmpeg is decoding the entire video frame in order to extract the caption data. There is a cbs_userdata patchset on the ffmpeg-devel mailing list which could be used to extract a53cc packets without the overhead of video decoding.

[Balling]

Is this still the case? What is cbs_userdata patchset and is it used for cc?

[Balling]

My tests show that cc decoders for h264 are now very very fast and do not decode main bitstream, only read T.35 SEI.