Opened 12 years ago
Last modified 3 years ago
#1778 reopened enhancement
EIA-608 / EIA-708 Closed Captions disappear when transcoding/reencoding
| Reported by: | Mike | Owned by: | Anshul |
|---|---|---|---|
| Priority: | wish | Component: | undetermined |
| Version: | git-master | Keywords: | cc |
| Cc: | Blocked By: | ||
| Blocking: | Reproduced by developer: | no | |
| Analyzed by developer: | no |
Description
Summary of the bug:
When transoding/reencoding video ffmepg loses the CC data that was embedded within the actual video stream itself. This type of CC is referred to as EIA-608/EIA-708 and is muxed to the video stream following guidelines in SCTE 128 from my research. If you use '-c:v copy' the CC remains intact. I'm capturing live video from the gige port of a Motorola DSR-6100 IRD that is putting out UDP multicast TS.
This PDF give more details regarding how this method of CC works:
http://www.evertz.com/resources/eia_608_708_cc.pdf
Here are the files produced from the below command. They are larger than the 10MB requested so I've hosted them on my site, here are the direct links.
http://mikecheat.com/disjrhd.ts - Original(18.2MB)
http://mikecheat.com/disjrsd.ts - Reencoded(8.9MB)
There is another open source project that seems to have figured out how to pull this CC data from video. They have the source code on their website, I've included the link.
http://zapping.sourceforge.net/ZVBI/index.html
How to reproduce:
root@hdmux:/home/mike# ffmpeg -i 'udp://239.1.1.3:6100?fifo_size=9000000' -map 0:p:1:0 -c:v mpeg2video -s 704x480 -r ntsc -b:v 3000k -map 0:p:1:1 -c:a mp3 -ac 2 -ar 48000 -b:a 128k -f mpegts disjrsd.ts -map 0:p:1:0 -c:v copy -map 0:p:1:1 -c:a copy -f mpegts disjrhd.ts
ffmpeg version 1.0 Copyright (c) 2000-2012 the FFmpeg developers
built on Sep 28 2012 14:24:44 with gcc 4.4.5 (Debian 4.4.5-8)
configuration: --enable-gpl --enable-nonfree --enable-shared --enable-runtime-cpudetect --enable-libmp3lame --enable-libx264
libavutil 51. 73.101 / 51. 73.101
libavcodec 54. 59.100 / 54. 59.100
libavformat 54. 29.104 / 54. 29.104
libavdevice 54. 2.101 / 54. 2.101
libavfilter 3. 17.100 / 3. 17.100
libswscale 2. 1.101 / 2. 1.101
libswresample 0. 15.100 / 0. 15.100
libpostproc 52. 0.100 / 52. 0.100
[mpegts @ 0x2169240] Unable to seek back to the start
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] mmco: unref short failure
Last message repeated 2 times
[mpegts @ 0x2169240] max_analyze_duration 5000000 reached at 5003333
[mpegts @ 0x2169240] Estimating duration from bitrate, this may be inaccurate
Input #0, mpegts, from 'udp://239.1.1.3:6100?fifo_size=9000000':
Duration: N/A, start: 11940.555644, bitrate: 768 kb/s
Program 1
Stream #0:0[0x1e00]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 61.76 fps, 59.94 tbr, 90k tbn, 119.88 tbc
Stream #0:1[0x1020](eng): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), s16, 384 kb/s
Stream #0:2[0x1021](spa): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, s16, 192 kb/s
Stream #0:3[0x1022](eng): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, s16, 192 kb/s
File 'disjrhd.ts' already exists. Overwrite ? [y/N] y
muxrate VBR, pcr every 2 pkts, sdt every 200, pat/pmt every 40 pkts
[mpegts @ 0x22aad60] muxrate VBR, pcr every 5 pkts, sdt every 200, pat/pmt every 40 pkts
Output #0, mpegts, to 'disjrsd.ts':
Metadata:
encoder : Lavf54.29.104
Stream #0:0: Video: mpeg2video, yuv420p, 704x480 [SAR 40:33 DAR 16:9], q=2-31, 3000 kb/s, 90k tbn, 29.97 tbc
Stream #0:1(eng): Audio: mp3, 48000 Hz, stereo, s16, 128 kb/s
Output #1, mpegts, to 'disjrhd.ts':
Metadata:
encoder : Lavf54.29.104
Stream #1:0: Video: h264 ([27][0][0][0] / 0x001B), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 61.76 fps, 90k tbn, 59.94 tbc
Stream #1:1(eng): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), 384 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (h264 -> mpeg2video)
Stream #0:1 -> #0:1 (ac3 -> libmp3lame)
Stream #0:0 -> #1:0 (copy)
Stream #0:1 -> #1:1 (copy)
Press [q] to stop, ? for help
[h264 @ 0x28a5740] Missing reference picture
[h264 @ 0x28a5740] decode_slice_header error
[h264 @ 0x28a5740] concealing 3600 DC, 3600 AC, 3600 MV errors in B frame
[h264 @ 0x28a63a0] Missing reference picture
[h264 @ 0x28a63a0] decode_slice_header error
[h264 @ 0x28a63a0] concealing 3600 DC, 3600 AC, 3600 MV errors in B frame
[h264 @ 0x28a68e0] reference picture missing during reorder
[h264 @ 0x28a68e0] Missing reference picture
[h264 @ 0x28a68e0] decode_slice_header error
[h264 @ 0x28a68e0] concealing 3600 DC, 3600 AC, 3600 MV errors in P frame
[h264 @ 0x28a7360] mmco: unref short failure
[h264 @ 0x2c4ddc0] mmco: unref short failure
frame= 656 fps= 47 q=2.0 Lq=-1.0 size= 9102kB time=00:00:21.93 bitrate=3398.6kbits/s dup=22 drop=628
video:24157kB audio:1326kB subtitle:0 global headers:0kB muxing overhead -64.284273%
Received signal 2: terminating.
You can't use my command exactly as is because I'm capturing from a live source. You can use one of the above files to emulate the problem. If any other info is needed please let me know.
Thank you
Change History (15)
comment:1 by , 12 years ago
comment:2 by , 12 years ago
| Component: | FFmpeg → undetermined |
|---|---|
| Keywords: | closed caption subtitle teletext eia 608 708 removed |
| Priority: | normal → wish |
| Status: | new → open |
| Type: | defect → enhancement |
| Version: | 1.0 → git-master |
FFmpeg does not support closed captions, see also tickett #846.
comment:3 by , 12 years ago
Wow all this time I was thinking it did. Made the assumption that subtitles and closed caption were the same thing. Okay well thanks for the update.
comment:4 by , 12 years ago
The situation is not quite as simple.
The problem in this specific case is that it does not have CC data in a separate stream but embedded in the video frame data.
FFmpeg currently has no way of handling this mess where one stream (video) contains another one (subtitles/CC).
CC in e.g. wtv, mxf and even mov as in ticket #846 is actually supported in so far as extracting it is possible, we just can't decode it into plain text or such and we can't mux it either.
comment:5 by , 12 years ago
I see. Could any of the code from the libzvbi project maybe help with this? It seems they've found a way to gather that information from the video stream and then mux it into a TS. There is also another project called OB-Encoder, they use libav, libx264, and libzvbi with a third party mpegts muxer to produce the desired output. I could use their program but they don't currently support mpeg2 and some of the more advance features of ffmepg that I've accustomed to.
libzvbi - http://zapping.sourceforge.net/ZVBI/Download.html
obencoder - http://code.google.com/p/open-broadcast-encoder/downloads/list
comment:6 by , 10 years ago
| Owner: | set to |
|---|
comment:7 by , 10 years ago
It is in pipeline of reviews, someone willing to review and comment please do so.
comment:8 by , 10 years ago
| Resolution: | → fixed |
|---|---|
| Status: | open → closed |
solved in 4b6262610bd516320b3fca71ef03ff9bf1aa5066
comment:9 by , 10 years ago
This is only for extracting EIA-607/708 captions from the input, correct? Any chance you can also add the ability to insert captions (in EIA-608/708 form) as well? I'd like to be able to produce an mpeg2 ts which contains EIA-608/708 captions instead of having them burned into the video.
comment:10 by , 10 years ago
| Resolution: | fixed |
|---|---|
| Status: | closed → reopened |
yes there is way, cc608 encoder need to be implemented
comment:11 by , 9 years ago
I'm not sure if this should constitute a new ticket, but the "lavfi" method in the revision doesn't work exactly right on a FinalCut-Captioned MOV file. FFMPEG didn't extract anything regardless of seeing the input track. Sample MOV available, but is 4.8GB and only 5mins long.
ffmpeg.exe -f lavfi -i "movie=OPEN_HOUSE_5275_WITH_CC_5mins.mov[out0+subcc]"
-map s "OPEN_HOUSE_5275_WITH_CC_5mins.mov.srt"
ffmpeg version N-73002-gf6c3f1e Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.9.2 (GCC)
configuration: --arch=x86_64 --target-os=mingw32 --cross-prefix=/mnt/st2/ffmpe
g_macosx/sandbox/mingw-w64-x86_64/bin/x86_64-w64-mingw32- --pkg-config=pkg-confi
g --enable-gpl --disable-decoder=jpeg2000 --enable-libsoxr --enable-fontconfig -
-enable-libass --enable-libutvideo --enable-libbluray --enable-iconv --enable-li
btwolame --extra-cflags=-DLIBTWOLAME_STATIC --enable-libzvbi --enable-libcaca --
enable-libmodplug --extra-libs=-lstdc++ --extra-libs=-lpng --enable-libvidstab -
-enable-libx265 --enable-decklink --extra-libs=-loleaut32 --enable-libx264 --ena
ble-libxvid --enable-libmp3lame --enable-version3 --enable-zlib --enable-librtmp
--enable-libvorbis --enable-libtheora --enable-libspeex --enable-libopenjpeg --
enable-gnutls --enable-libgsm --enable-libfreetype --enable-libopus --disable-w3
2threads --enable-frei0r --enable-filter=frei0r --enable-libvo-aacenc --enable-b
zlib --enable-libxavs --enable-libopencore-amrnb --enable-libopencore-amrwb --en
able-libvo-amrwbenc --enable-libschroedinger --enable-libvpx --enable-libilbc --
enable-libwavpack --enable-libwebp --enable-libgme --enable-dxva2 --enable-libdc
adec --enable-avisynth --enable-libmfx --enable-static --disable-shared --extra-
cflags= --prefix=/mnt/st2/ffmpeg_macosx/sandbox/mingw-w64-x86_64/x86_64-w64-ming
w32 --extra-libs=-lpsapi --enable-nonfree --enable-libfdk-aac --disable-libfaac
--enable-nvenc --enable-runtime-cpudetect
libavutil 54. 27.100 / 54. 27.100
libavcodec 56. 41.100 / 56. 41.100
libavformat 56. 37.100 / 56. 37.100
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 17.100 / 5. 17.100
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.100 / 1. 2.100
libpostproc 53. 3.100 / 53. 3.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 00000000004183a0] Could not find codec parameters for
stream 3 (Subtitle: none (c708 / 0x38303763), 1920x1080, 23 kb/s): unknown code
c
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, lavfi, from 'movie=OPEN_HOUSE_5275_WITH_CC_5mins.mov[out0+subcc]':
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0: Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le, 1920x108
0 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 30k tbn, 30k tbc
Stream #0:1: Subtitle: eia_608
File 'OPEN_HOUSE_5275_WITH_CC_5mins.mov.srt' already exists. Overwrite ? [y/N] y
Output #0, srt, to 'OPEN_HOUSE_5275_WITH_CC_5mins.mov.srt':
Metadata:
encoder : Lavf56.37.100
Stream #0:0: Subtitle: subrip (srt)
Metadata:
encoder : Lavc56.41.100 srt
Stream mapping:
Stream #0:1 -> #0:0 (eia_608 (cc_dec) -> subrip (srt))
Press [q] to stop, [?] for help
size= 0kB time=00:00:00.00 bitrate=N/A
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing ove
rhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters i
f used)
Without lavfi, you see a c708 codec detection:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'OPEN_HOUSE_5275_WITH_CC_5mins.mov':
Metadata:
major_brand : qt
minor_version : 537199360
compatible_brands: qt
creation_time : 2015-04-15 21:12:03
Duration: 00:05:00.03, start: 0.000000, bitrate: 138315 kb/s
Stream #0:0(und): Video: prores (apcn / 0x6E637061), yuv422p10le(bt709), 192
0x1080, 135984 kb/s, SAR 1:1 DAR 16:9, 29.97 fps, 29.97 tbr, 30k tbn, 30k tbc (d
efault)
Metadata:
creation_time : 2015-04-15 21:12:03
handler_name : Apple Alias Data Handler
encoder : Apple ProRes 422
timecode : 00:59:00;02
Stream #0:1(und): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, stereo, s3
2 (24 bit), 2304 kb/s (default)
Metadata:
creation_time : 2015-04-15 21:12:04
handler_name : Apple Alias Data Handler
Stream #0:2(und): Data: none (tmcd / 0x64636D74) (default)
Metadata:
creation_time : 2015-04-15 21:12:04
handler_name : Apple Alias Data Handler
timecode : 00:59:00;02
Stream #0:3(eng): Subtitle: none (c708 / 0x38303763), 1920x1080, 23 kb/s (de
fault)
Metadata:
creation_time : 2015-04-15 21:12:04
handler_name : Apple Alias Data Handler
At least one output file must be specified
I can't even do a "rawvideo" or -c:s copy on that stream to anywhere.
comment:12 by , 8 years ago
If the source has only single CC in 608/708 format then latest FFmpeg have the option to retain the 608/70d data after transcoding.
working file Media Info
Text #1 ID : 256 (0x100)-CC1 Menu ID : 1 (0x1) Format : EIA-608 Muxing mode : SCTE 128 / DTVCC Transport Muxing mode, more info : Muxed in Video #1 Duration : 20s 938ms Bit rate mode : Constant Stream size : 0.00 Byte (0%) Text #2 ID : 256 (0x100)-1 Menu ID : 1 (0x1) Format : EIA-708 Muxing mode : SCTE 128 / DTVCC Transport Muxing mode, more info : Muxed in Video #1 Duration : 20s 938ms Bit rate mode : Constant Stream size : 0.00 Byte (0%)
At the same time if the source has multiple CC streams then FFmpeg is not retaining the CC.
Not working file Media Info
Text #1 ID : 256 (0x100)-CC1 Menu ID : 1 (0x1) Format : EIA-608 Muxing mode : SCTE 128 / DTVCC Transport Muxing mode, more info : Muxed in Video #1 Duration : 26s 593ms Bit rate mode : Constant Stream size : 0.00 Byte (0%) Text #2 ID : 256 (0x100)-CC3 Menu ID : 1 (0x1) Format : EIA-608 Muxing mode : SCTE 128 / DTVCC Transport Muxing mode, more info : Muxed in Video #1 Duration : 26s 593ms Bit rate mode : Constant Stream size : 0.00 Byte (0%) Text #3 ID : 256 (0x100)-1 Menu ID : 1 (0x1) Format : EIA-708 Muxing mode : SCTE 128 / DTVCC Transport Muxing mode, more info : Muxed in Video #1 Duration : 26s 593ms Bit rate mode : Constant Stream size : 0.00 Byte (0%) Text #4 ID : 256 (0x100)-2 Menu ID : 1 (0x1) Format : EIA-708 Muxing mode : SCTE 128 / DTVCC Transport Muxing mode, more info : Muxed in Video #1 Duration : 26s 593ms Bit rate mode : Constant Stream size : 0.00 Byte (0%)
comment:13 by , 7 years ago
hemube, can you give an example of syntax required to make the above sample work? Trying to look in to this. We eventually would like the ability to use the decklink input sending in both video and 608 captions to be encoded into both rtmp outputs and hls outputs. Thanks.
comment:14 by , 4 years ago
Very important patch not applied https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210413143510.93256-1-dhanishvijayan@gmail.com/
comment:15 by , 3 years ago
Also this patch series should be applied too: https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=1057
PS I can provide as many samples as needed. This is only one feed of about 30 that I have access to.