Opened 4 years ago

Closed 3 years ago

#4550 closed defect (invalid)

Overlaying VOBSUB from MKV sample while deinterlacing produces various timing oddities

Reported by: 11rcombs Owned by:
Priority: normal Component: undetermined
Version: git-master Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:
Sample at http://puu.sh/hKq7C/513a386d04.mkv
Tweaking the output codec and container used in the command below gives a variety of unusual outputs, and behavior varies between players. I used libx264+mpegts for the console test because it's the case the problem was pointed out to me on, and because its symptoms are possibly the strangest out of what I tested (libx264, mpeg2video, mpeg4; mpegts, mp4, matroska). This case produces very juddery video with frames apparently dropped.

How to reproduce:

$ ffmpeg -i input.mkv -filter_complex '[0:0]yadif,[0:4]overlay[0]' -map '[0]' -codec:0 libx264 -preset ultrafast test.ts -loglevel verbose
ffmpeg version N-71845-g0b9aad1 Copyright (c) 2000-2015 the FFmpeg developers
  built with Apple LLVM version 6.1.0 (clang-602.0.49) (based on LLVM 3.6.0svn)
  configuration: --extra-cflags='-march=native' --disable-optimizations --disable-stripping --enable-debug --enable-libx265 --enable-gpl --enable-libopus --enable-version3 --enable-nonfree --enable-postproc --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-securetransport --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --prefix=/usr/local --enable-static --enable-shared --enable-avresample --enable-vda --enable-opencl --enable-libzvbi --cc=clang
  libavutil      54. 23.101 / 54. 23.101
  libavcodec     56. 35.101 / 56. 35.101
  libavformat    56. 31.100 / 56. 31.100
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 14.100 /  5. 14.100
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
[matroska,webm @ 0x7ffa5c00da00] parser not found for codec pcm_s16le, packets or times may be invalid.
    Last message repeated 1 times
Guessed Channel Layout for  Input Stream #0.3 : stereo
Input #0, matroska,webm, from 'input.mkv':
  Metadata:
    encoder         : libebml v1.3.1 + libmatroska v1.4.2
    creation_time   : 2015-05-04 21:31:23
  Duration: 00:00:20.27, start: 0.000000, bitrate: 6220 kb/s
    Chapter #0:0: start 0.080000, end 20.235000
    Metadata:
      title           : Chapter 01
    Stream #0:0(eng): Video: mpeg2video (Main), 1 reference frame, yuv420p(tv), 720x576 [SAR 16:15 DAR 4:3], max. 8000 kb/s, 25 fps, 25 tbr, 1k tbn, 50 tbc
    Metadata:
      _STATISTICS_WRITING_APP: mkvmerge v7.8.0 ('River Man') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v7.8.0 ('River Man') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2015-05-04 21:31:23
      _STATISTICS_WRITING_DATE_UTC-eng: 2015-05-04 21:31:23
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      BPS             : 4315022
      BPS-eng         : 4315022
      DURATION        : 00:00:20.160000000
      DURATION-eng    : 00:00:20.160000000
      NUMBER_OF_FRAMES: 504
      NUMBER_OF_FRAMES-eng: 504
      NUMBER_OF_BYTES : 10873857
      NUMBER_OF_BYTES-eng: 10873857
    Stream #0:1(eng): Audio: aac (LC), 48000 Hz, stereo, fltp
    Metadata:
      title           : Stereo
      _STATISTICS_WRITING_APP: mkvmerge v7.8.0 ('River Man') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v7.8.0 ('River Man') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2015-05-04 21:31:23
      _STATISTICS_WRITING_DATE_UTC-eng: 2015-05-04 21:31:23
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      BPS             : 163357
      BPS-eng         : 163357
      DURATION        : 00:00:20.160000000
      DURATION-eng    : 00:00:20.160000000
      NUMBER_OF_FRAMES: 945
      NUMBER_OF_FRAMES-eng: 945
      NUMBER_OF_BYTES : 411662
      NUMBER_OF_BYTES-eng: 411662
    Stream #0:2(eng): Audio: ac3, 48000 Hz, stereo, fltp, 224 kb/s (default)
    Metadata:
      title           : Stereo
      _STATISTICS_WRITING_APP: mkvmerge v7.8.0 ('River Man') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v7.8.0 ('River Man') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2015-05-04 21:31:23
      _STATISTICS_WRITING_DATE_UTC-eng: 2015-05-04 21:31:23
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      BPS             : 224000
      BPS-eng         : 224000
      DURATION        : 00:00:20.192000000
      DURATION-eng    : 00:00:20.192000000
      NUMBER_OF_FRAMES: 631
      NUMBER_OF_FRAMES-eng: 631
      NUMBER_OF_BYTES : 565376
      NUMBER_OF_BYTES-eng: 565376
    Stream #0:3(eng): Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s
    Metadata:
      title           : Stereo
      _STATISTICS_WRITING_APP: mkvmerge v7.8.0 ('River Man') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v7.8.0 ('River Man') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2015-05-04 21:31:23
      _STATISTICS_WRITING_DATE_UTC-eng: 2015-05-04 21:31:23
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      BPS             : 1536050
      BPS-eng         : 1536050
      DURATION        : 00:00:20.266000000
      DURATION-eng    : 00:00:20.266000000
      NUMBER_OF_FRAMES: 608
      NUMBER_OF_FRAMES-eng: 608
      NUMBER_OF_BYTES : 3891200
      NUMBER_OF_BYTES-eng: 3891200
    Stream #0:4(eng): Subtitle: dvd_subtitle, 720x576 (default)
    Metadata:
      _STATISTICS_WRITING_APP: mkvmerge v7.8.0 ('River Man') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v7.8.0 ('River Man') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2015-05-04 21:31:23
      _STATISTICS_WRITING_DATE_UTC-eng: 2015-05-04 21:31:23
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      BPS             : 0
      BPS-eng         : 0
      DURATION        : 00:00:00.000000000
      DURATION-eng    : 00:00:00.000000000
      NUMBER_OF_FRAMES: 0
      NUMBER_OF_FRAMES-eng: 0
      NUMBER_OF_BYTES : 0
      NUMBER_OF_BYTES-eng: 0
[graph 0 input from stream 0:0 @ 0x7ffa5bf00880] w:720 h:576 pixfmt:yuv420p tb:1/1000 fr:25/1 sar:16/15 sws_param:flags=2
[graph 0 input from stream 0:4 @ 0x7ffa5bf00b80] w:720 h:576 pixfmt:bgra tb:1/1000 fr:0/1 sar:0/1 sws_param:flags=2
[auto-inserted scaler 0 @ 0x7ffa5bc22ac0] w:iw h:ih flags:'bilinear' interl:0
[Parsed_overlay_1 @ 0x7ffa5d800560] auto-inserting filter 'auto-inserted scaler 0' between the filter 'graph 0 input from stream 0:4' and the filter 'Parsed_overlay_1'
[auto-inserted scaler 1 @ 0x7ffa5bc22e60] w:iw h:ih flags:'bilinear' interl:0
[Parsed_overlay_1 @ 0x7ffa5d800560] auto-inserting filter 'auto-inserted scaler 1' between the filter 'Parsed_yadif_0' and the filter 'Parsed_overlay_1'
[auto-inserted scaler 0 @ 0x7ffa5bc22ac0] w:720 h:576 fmt:bgra sar:0/1 -> w:720 h:576 fmt:yuv420p sar:0/1 flags:0x2
[auto-inserted scaler 1 @ 0x7ffa5bc22e60] w:720 h:576 fmt:yuv420p sar:16/15 -> w:720 h:576 fmt:yuva420p sar:16/15 flags:0x2
[Parsed_overlay_1 @ 0x7ffa5d800560] main w:720 h:576 fmt:yuv420p overlay w:720 h:576 fmt:yuva420p eof_action:repeat
[Parsed_overlay_1 @ 0x7ffa5d800560] [framesync @ 0x7ffa5e000028] Selected 1/2000 time base
[Parsed_overlay_1 @ 0x7ffa5d800560] [framesync @ 0x7ffa5e000028] Sync level 2
[libx264 @ 0x7ffa5c804e00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 0x7ffa5c804e00] profile Constrained Baseline, level 5.2
[mpegts @ 0x7ffa5c804200] muxrate VBR, pcr every 100 pkts, sdt every 200, pat/pmt every 40 pkts
Output #0, mpegts, to 'test.ts':
  Metadata:
    encoder         : Lavf56.31.100
    Chapter #0:0: start 0.080000, end 20.235000
    Metadata:
      title           : Chapter 01
    Stream #0:0: Video: h264 (libx264), -1 reference frame, yuv420p, 720x576, q=-1--1, 90k tbn, 1k tbc (default)
    Metadata:
      encoder         : Lavc56.35.101 libx264
Stream mapping:
  Stream #0:0 (mpeg2video) -> yadif
  Stream #0:4 (dvdsub) -> overlay:main
  overlay -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
*** 197 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
*** 79 dup! fps=0.0 q=17.0 size=     457kB time=00:00:00.98 bitrate=3799.7kbits/s dup=990 drop=0    
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
*** 79 dup! fps=2018 q=17.0 size=    1043kB time=00:00:02.06 bitrate=4134.2kbits/s dup=2061 drop=0    
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=1998 q=17.0 size=    1951kB time=00:00:03.06 bitrate=5214.0kbits/s dup=3053 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=2031 q=17.0 size=    2887kB time=00:00:04.14 bitrate=5704.0kbits/s dup=4124 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=2034 q=17.0 size=    3803kB time=00:00:05.22 bitrate=5961.6kbits/s dup=5195 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=2036 q=17.0 size=    4818kB time=00:00:06.30 bitrate=6259.5kbits/s dup=6266 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=2038 q=17.0 size=    5794kB time=00:00:07.38 bitrate=6426.3kbits/s dup=7337 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=2048 q=17.0 size=    6758kB time=00:00:08.46 bitrate=6539.4kbits/s dup=8408 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=2049 q=17.0 size=    7580kB time=00:00:09.54 bitrate=6504.6kbits/s dup=9479 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=2053 q=17.0 size=    8479kB time=00:00:10.62 bitrate=6537.0kbits/s dup=10550 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=2052 q=17.0 size=    9465kB time=00:00:11.70 bitrate=6623.6kbits/s dup=11621 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=2057 q=17.0 size=   10613kB time=00:00:12.78 bitrate=6800.0kbits/s dup=12692 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=2057 q=17.0 size=   11624kB time=00:00:13.86 bitrate=6867.4kbits/s dup=13763 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=2059 q=17.0 size=   12670kB time=00:00:14.94 bitrate=6944.3kbits/s dup=14834 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=2054 q=17.0 size=   13825kB time=00:00:16.02 bitrate=7066.9kbits/s dup=15905 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=2058 q=17.0 size=   14883kB time=00:00:17.10 bitrate=7127.3kbits/s dup=16976 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=2058 q=17.0 size=   15777kB time=00:00:18.18 bitrate=7107.0kbits/s dup=18047 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup! fps=2062 q=17.0 size=   16793kB time=00:00:19.26 bitrate=7140.4kbits/s dup=19118 drop=0    
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
    Last message repeated 1 times
*** 199 dup!
*** 79 dup!
[Parsed_overlay_1 @ 0x7ffa5d800560] [framesync @ 0x7ffa5e000028] Sync level 1
[Parsed_overlay_1 @ 0x7ffa5d800560] [framesync @ 0x7ffa5e000028] Sync level 0
No more output streams to write to, finishing.
frame=20081 fps=2058 q=-1.0 Lsize=   17525kB time=00:00:20.08 bitrate=7149.1kbits/s dup=19911 drop=0    
video:14257kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 22.922169%
Input file #0 (input.mkv):
  Input stream #0:0 (video): 504 packets read (10873857 bytes); 504 frames decoded; 
  Input stream #0:1 (audio): 8 packets read (2771 bytes); 
  Input stream #0:2 (audio): 8 packets read (7168 bytes); 
  Input stream #0:3 (audio): 8 packets read (51200 bytes); 
  Input stream #0:4 (subtitle): 0 packets read (0 bytes); 0 frames decoded; 
  Total: 528 packets (10934996 bytes) demuxed
Output file #0 (test.ts):
  Output stream #0:0 (video): 20081 frames encoded; 20081 packets muxed (14598802 bytes); 
  Total: 20081 packets (14598802 bytes) muxed
[libx264 @ 0x7ffa5c804e00] frame I:81    Avg QP:14.33  size: 50080
[libx264 @ 0x7ffa5c804e00] frame P:20000 Avg QP:17.34  size:   527
[libx264 @ 0x7ffa5c804e00] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0x7ffa5c804e00] mb P  I16..4:  0.4%  0.0%  0.0%  P16..4:  9.0%  0.0%  0.0%  0.0%  0.0%    skip:90.6%
[libx264 @ 0x7ffa5c804e00] coded y,uvDC,uvAC intra: 51.3% 24.9% 9.1% inter: 1.1% 6.7% 0.0%
[libx264 @ 0x7ffa5c804e00] i16 v,h,dc,p: 22% 44% 19% 16%
[libx264 @ 0x7ffa5c804e00] i8c dc,h,v,p: 53% 27% 16%  4%
[libx264 @ 0x7ffa5c804e00] kb/s:5815.97

Change History (5)

comment:1 Changed 4 years ago by cehoyos

Is the issue also reproducible with a native encoder like mpeg4 or mpeg2video?

comment:2 Changed 4 years ago by 11rcombs

Yes (this is mentioned in my original description); the result and logs just come out a bit differently.

comment:3 Changed 4 years ago by 11rcombs

Looks like vf_yadif.c changes the time base (in config_props), and vf_overlay.c uses the time base it gets in config_output, which doesn't get updated. Not sure if this is overlay's fault for using the wrong thing, yadif's fault for changing the wrong thing (or changing at the wrong time), or lavfi's fault for not passing something around correctly.

comment:4 Changed 4 years ago by 11rcombs

Issue also occurs with kerndeint instead of yadif, which suggests this is either an overlay or lavfi problem.

comment:5 Changed 3 years ago by cehoyos

  • Resolution set to invalid
  • Status changed from new to closed

The syntax is wrong, should be: yadif[background],[background][0:4]overlay
Note that this would have been more obvious if the sample you provided actually showed subtitles.

Note: See TracTickets for help on using tickets.