Opened 2 years ago

Closed 2 years ago

#8723 closed defect (fixed)

scc captions overlap if more than one code to erase display per line

Reported by: dave rice Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: scc
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

I am using
ffmpeg -f lavfi -i color=s=720x480 -vf subtitles=URBN02008_SEG01_UUAPRIL2020_SHOW_OPEN_TOSS.scc -t 30 -pix_fmt yuv420p -y test.mp4

to produce an output with hard-coded subtitles from an scc file (I'm recreating this with a generated video source since the issue isn't related to the source file used). Some lines htat include more than code to erase the onscreen display and continue with more characters (at 20 seconds in with this example).

How to reproduce:

ffmpeg -f lavfi -i color=s=720x480 -vf subtitles=URBN02008_SEG01_UUAPRIL2020_SHOW_OPEN_TOSS.scc -t 30 -pix_fmt yuv420p -y test.mp4
ffmpeg version 4.2.3 Copyright (c) 2000-2020 the FFmpeg developers
  built with Apple clang version 11.0.3 (clang-1103.0.32.59)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.3_1 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags=-fno-stack-check --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, lavfi, from 'color=s=720x480':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 720x480 [SAR 1:1 DAR 3:2], 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[Parsed_subtitles_0 @ 0x7fd5b8c04880] Shaper: FriBidi 0.19.7 (SIMPLE) HarfBuzz-ng 2.6.6 (COMPLEX)
[Parsed_subtitles_0 @ 0x7fd5b8c04880] Using font provider coretext
[libx264 @ 0x7fd5ba014a00] using SAR=1/1
[libx264 @ 0x7fd5ba014a00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7fd5ba014a00] profile High, level 3.0, 4:2:0, 8-bit
[libx264 @ 0x7fd5ba014a00] 264 - core 159 r2999 296494a - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'test.mp4':
  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 720x480 [SAR 1:1 DAR 3:2], q=-1--1, 25 fps, 12800 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.54.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
[Parsed_subtitles_0 @ 0x7fd5b8c04880] fontselect: (Monospace, 400, 0) -> /System/Library/Fonts/Courier.dfont, -1, Courier
frame=  750 fps=684 q=-1.0 Lsize=      74kB time=00:00:29.88 bitrate=  20.2kbits/s speed=27.3x    
video:64kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 15.011199%
[libx264 @ 0x7fd5ba014a00] frame I:7     Avg QP: 9.91  size:  3949
[libx264 @ 0x7fd5ba014a00] frame P:187   Avg QP: 9.56  size:   122
[libx264 @ 0x7fd5ba014a00] frame B:556   Avg QP:12.67  size:    26
[libx264 @ 0x7fd5ba014a00] consecutive B-frames:  0.9%  0.3%  1.2% 97.6%
[libx264 @ 0x7fd5ba014a00] mb I  I16..4: 75.7% 19.6%  4.7%
[libx264 @ 0x7fd5ba014a00] mb P  I16..4:  0.0%  0.0%  0.1%  P16..4:  0.1%  0.0%  0.0%  0.0%  0.0%    skip:99.7%
[libx264 @ 0x7fd5ba014a00] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  0.3%  0.0%  0.0%  direct: 0.0%  skip:99.7%  L0:53.4% L1:46.6% BI: 0.0%
[libx264 @ 0x7fd5ba014a00] 8x8 transform intra:19.7% inter:1.9%
[libx264 @ 0x7fd5ba014a00] coded y,uvDC,uvAC intra: 3.8% 0.0% 0.0% inter: 0.0% 0.0% 0.0%
[libx264 @ 0x7fd5ba014a00] i16 v,h,dc,p: 95%  1%  4%  0%
[libx264 @ 0x7fd5ba014a00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 70% 11% 19%  0%  0%  0%  0%  0%  0%
[libx264 @ 0x7fd5ba014a00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 14% 31%  3%  3%  6%  5%  4%  3%
[libx264 @ 0x7fd5ba014a00] i8c dc,h,v,p: 100%  0%  0%  0%
[libx264 @ 0x7fd5ba014a00] Weighted P-Frames: Y:1.6% UV:0.0%
[libx264 @ 0x7fd5ba014a00] ref P L0: 77.7%  6.1% 15.5%  0.7%
[libx264 @ 0x7fd5ba014a00] ref B L0: 55.3% 44.7%
[libx264 @ 0x7fd5ba014a00] ref B L1: 99.7%  0.3%
[libx264 @ 0x7fd5ba014a00] kb/s:17.32

At the 20 second mark of the output the text on the output frames shows
"BUT FIRST...E ON BLACK \nCORALS AND PRESERVING EELS."

If the process is done by first converting the scc to an srt like:

ffmpeg -i URBN02008_SEG01_UUAPRIL2020_SHOW_OPEN_TOSS.scc URBN02008_SEG01_UUAPRIL2020_SHOW_OPEN_TOSS.srt
 ffmpeg -f lavfi -i color=s=720x480 -vf subtitles=URBN02008_SEG01_UUAPRIL2020_SHOW_OPEN_TOSS.srt -t 30 -pix_fmt yuv420p -y test2.mp4

Then that resulting frame shows:
"RESEARCH DONE ON BLACK \nCORALS AND PRESERVING EELS. BUT FIRST..."

Which is a little better since the text is unobscured, but the instructions of the scc suggest.
"RESEARCH DONE ON BLACK \nCORALS AND PRESERVING EELS."
<clear on-screen text>
"BUT FIRST..."

Attachments (1)

URBN02008_SEG01_UUAPRIL2020_SHOW_OPEN_TOSS.scc (1.8 KB ) - added by dave rice 2 years ago.

Download all attachments as: .zip

Change History (4)

comment:1 by dave rice, 2 years ago

I re-ran this with git-master, as the last test used 4.2.3. The output had the same overlapping text issue but the timing of the presentations of the text was slightly different. With git-master the output is:

ffmpeg -f lavfi -i color=s=720x480 -vf subtitles=URBN02008_SEG01_UUAPRIL2020_SHOW_OPEN_TOSS.scc -t 30 -pix_fmt yuv420p -y test_git_master.mp4
ffmpeg version git-2020-06-11-0b182ff Copyright (c) 2000-2020 the FFmpeg developers
  built with Apple clang version 11.0.3 (clang-1103.0.32.62)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD-0b182ff_1 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags=-fno-stack-check --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 54.100 / 56. 54.100
  libavcodec     58. 92.100 / 58. 92.100
  libavformat    58. 46.101 / 58. 46.101
  libavdevice    58. 11.100 / 58. 11.100
  libavfilter     7. 86.100 /  7. 86.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
Input #0, lavfi, from 'color=s=720x480':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 720x480 [SAR 1:1 DAR 3:2], 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[Parsed_subtitles_0 @ 0x7fa602d0e900] Shaper: FriBidi 0.19.7 (SIMPLE) HarfBuzz-ng 2.6.6 (COMPLEX)
[Parsed_subtitles_0 @ 0x7fa602d0e900] Using font provider coretext
[libx264 @ 0x7fa60380ea00] using SAR=1/1
[libx264 @ 0x7fa60380ea00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7fa60380ea00] profile High, level 3.0, 4:2:0, 8-bit
[libx264 @ 0x7fa60380ea00] 264 - core 159 r2999 296494a - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'test_git_master.mp4':
  Metadata:
    encoder         : Lavf58.46.101
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 720x480 [SAR 1:1 DAR 3:2], q=-1--1, 25 fps, 12800 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.92.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
[Parsed_subtitles_0 @ 0x7fa602d0e900] fontselect: (Monospace, 400, 0) -> /System/Library/Fonts/Courier.dfont, -1, Courier
frame=  750 fps=636 q=-1.0 Lsize=      74kB time=00:00:29.88 bitrate=  20.2kbits/s speed=25.3x    
video:64kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 15.043424%
[libx264 @ 0x7fa60380ea00] frame I:7     Avg QP: 9.96  size:  3935
[libx264 @ 0x7fa60380ea00] frame P:187   Avg QP: 9.57  size:   122
[libx264 @ 0x7fa60380ea00] frame B:556   Avg QP:12.67  size:    26
[libx264 @ 0x7fa60380ea00] consecutive B-frames:  0.9%  0.3%  1.2% 97.6%
[libx264 @ 0x7fa60380ea00] mb I  I16..4: 75.7% 19.6%  4.7%
[libx264 @ 0x7fa60380ea00] mb P  I16..4:  0.0%  0.0%  0.1%  P16..4:  0.1%  0.0%  0.0%  0.0%  0.0%    skip:99.7%
[libx264 @ 0x7fa60380ea00] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  0.3%  0.0%  0.0%  direct: 0.0%  skip:99.7%  L0:53.2% L1:46.8% BI: 0.0%
[libx264 @ 0x7fa60380ea00] 8x8 transform intra:19.7% inter:2.7%
[libx264 @ 0x7fa60380ea00] coded y,uvDC,uvAC intra: 3.8% 0.0% 0.0% inter: 0.0% 0.0% 0.0%
[libx264 @ 0x7fa60380ea00] i16 v,h,dc,p: 95%  1%  4%  0%
[libx264 @ 0x7fa60380ea00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 69% 12% 19%  0%  0%  0%  0%  0%  0%
[libx264 @ 0x7fa60380ea00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 14% 31%  3%  3%  6%  5%  4%  4%
[libx264 @ 0x7fa60380ea00] i8c dc,h,v,p: 100%  0%  0%  0%
[libx264 @ 0x7fa60380ea00] Weighted P-Frames: Y:1.6% UV:0.0%
[libx264 @ 0x7fa60380ea00] ref P L0: 77.5%  6.2% 15.6%  0.7%
[libx264 @ 0x7fa60380ea00] ref B L0: 55.3% 44.7%
[libx264 @ 0x7fa60380ea00] ref B L1: 99.7%  0.3%
[libx264 @ 0x7fa60380ea00] kb/s:17.29

With this output at 24.120 it shows this text:

"BUT FIRST...E ON BLACK \nCORALS AND PRESERVING EELS."

In the scc I see:
00:00:19:05 9420 942c 942f 9420 13d0 5245 d345 c152 43c8 20c4 4fce 4520 4fce 20c2 4cc1 43cb 1370 434f 52c1 4cd3 20c1 cec4 20d0 5245 d345 52d6 49ce c720 4545 4cd3 ae80

"9420 942c 942f 9420" clears the text from the screen, places the prior caption on screen (which is ">>> YES, THE ROOF POD."), and then resumes filling a caption buffer with the subsequent codes which position and provide the text of "RESEARCH DONE ON BLACK CORALS AND PRESERVING EELS."

The '942f' code (End Of Caption: Swap off-screen buffer with caption screen.) at 00:00:19:05 of the scc works correctly and at 19.200 ">>> YES, THE ROOF POD." is shown.

The next scc line is
00:00:20:19 9420 942c 942f

This is supposed to clear the text from the screen and then swap the buffer (filled with "RESEARCH DONE ON BLACK CORALS AND PRESERVING EELS.") to be on-screen. In the output file at 20.640 the text on-screen is cleared via 942c. But the 942f command to swap the buffered text with the on-screen text doesn't happen. "RESEARCH DONE ON BLACK CORALS AND PRESERVING EELS." should appear at 20.640 but does not.

The next scc line is
00:00:20:25 9420 13d0 c2d5 5420 4649 52d3 54ae aeae

which resets the position of the text to row 12, column 00 and filled the buffer with "BUT FIRST...". Since "RESEARCH DONE ON BLACK CORALS AND PRESERVING EELS." is still in the buffer and not displayed, the "BUT FIRST..." is written on top of it while in the buffer, so the buffer now holds "BUT FIRST...E ON BLACK CORALS AND PRESERVING EELS."

The next scc line starts with

00:00:24:03 9420 942c 942f 9420

which clears the on-screen text (but in ffmpeg there isn't any at this time because "RESEARCH DONE ON BLACK CORALS AND PRESERVING EELS." was never shown as should have occurred via the scc line at 00:00:20:19. Then the command at 942f places the buffered text on screen, so at 24.120
"BUT FIRST...E ON BLACK CORALS AND PRESERVING EELS." is shown.

The expectation is that:

19.200 ">>> YES, THE ROOF POD."
20.627 "RESEARCH DONE ON BLACK CORALS AND PRESERVING EELS."
24.120 "BUT FIRST..."

but the output is:
19.200 ">>> YES, THE ROOF POD."
24.120 "BUT FIRST...E ON BLACK CORALS AND PRESERVING EELS."

comment:2 by Elon Musk, 2 years ago

Your explanation is unfortunately invalid, the clearing works fine it just get mixed next subtitle line with old one. If you convert to .ass subtitle format you will notice corrupt last line with items that are missing timestamp.

If one split last line from uploaded scc at next 9420 942c 942f one gets intended output.

comment:3 by Elon Musk, 2 years ago

Component: avcodecavformat
Resolution: fixed
Status: newclosed
Note: See TracTickets for help on using tickets.