Opened 3 years ago

Closed 2 years ago

Last modified 2 years ago

#4990 closed defect (invalid)

fieldmatch+decimate duplicate frames with yuv

Reported by: nullb Owned by:
Priority: normal Component: avfilter
Version: git-master Keywords: pullup
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug:
FFMPEG cannot currently fieldmatch and decimate a yuv input correctly. Resulting in duplicate frames from either mismatched fields or a failure to apply a proper decimate pattern.

The following AVS script using the same source which I've attached for your convenience does produce a correct result without duplicate frames, as fieldmatch+decimate are meant to be based on TIVTC I was expecting the same fault but the AVS script succeeds.

LoadPlugin("rawsource.dll")
LoadPlugin("TIVTC.dll")

RawSource("bugged.yuv", 1920, 1080, "I420").AssumeFPS("ntsc_video").AssumeTFF()
TFM().TDecimate()

Plugins for AVS are:
http://www.avisynth.nl/users/warpenterprises/files/rawsource_5F25_dll_20060728.zip
http://bengal.missouri.edu/~kes25c/TIVTCv105.zip

Source in yuv format:
https://drive.google.com/file/d/0B2keH696oskMYWVNSW5McFRxTTA/view?usp=sharing

How to reproduce:

% ffmpeg -pix_fmt yuv420p -s 1920x1080 -r 30000/1001 -i bugged.yuv -vf "setfield=mode=tff,fieldmatch,decimate" test.mkv

You will find duplicate frames on frames 2,3 - The first frame is also incorrect as you'll see in comparison to AVS.
I have only attached a small sample as yuv frames are obviously quite large but on the complete yuv dump multiple duplicate frames appear.
It still happens even with the use of a stronger set of filter parameters e.g: fieldmatch=order=tff:combmatch=full,yadif=deint=interlaced,decimate

Bug is present on the git-master and all versions of ffmpeg since fieldmatch+decimate was added, unfortunately I'm not familiar enough with the avfilter to create a solution, hopefully somebody here can.

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

Change History (9)

comment:1 Changed 3 years ago by cehoyos

  • Keywords pullup added; decimate duplicate frames bug removed
  • Priority changed from critical to normal

Please provide the command line that allows to reproduce the issue together with the complete, uncut console output to make this a valid ticket.

comment:2 Changed 3 years ago by cehoyos

  • Reproduced by developer set
  • Status changed from new to open

For future tickets, please provide the command line that allows to reproduce the issue together with the complete, uncut console output.

$ ffmpeg -loglevel debug -i fieldmatched.avi -vf decimate -f null -
ffmpeg version N-77670-g962727a Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --enable-libfreetype --enable-fontconfig --enable-libfribidi
  libavutil      55. 12.100 / 55. 12.100
  libavcodec     57. 21.100 / 57. 21.100
  libavformat    57. 21.100 / 57. 21.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6. 23.100 /  6. 23.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-i' ... matched as input file with argument 'fieldmatched.avi'.
Reading option '-vf' ... matched as option 'vf' (set video filters) with argument 'decimate'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'null'.
Reading option '-' ... matched as output file.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input file fieldmatched.avi.
Successfully parsed a group of options.
Opening an input file: fieldmatched.avi.
[avi @ 0x32bc400] Format avi probed with size=2048 and score=100
[avi @ 0x32bcb80] use odml:1
[avi @ 0x32bc400] Before avformat_find_stream_info() pos: 5720 bytes read:98952 seeks:4
[avi @ 0x32bc400] parser not found for codec ffv1, packets or times may be invalid.
    Last message repeated 1 times
[avi @ 0x32bc400] All info found
[avi @ 0x32bc400] After avformat_find_stream_info() pos: 635993 bytes read:696457 seeks:4 frames:1
Input #0, avi, from 'fieldmatched.avi':
  Metadata:
    encoder         : Lavf57.20.100
  Duration: 00:00:00.67, start: 0.000000, bitrate: 123059 kb/s
    Stream #0:0, 1, 1001/30000: Video: ffv1, 1 reference frame (FFV1 / 0x31564646), yuv420p, 1920x1080, 1001/30000, 129457 kb/s, 29.97 fps, 29.97 tbr, 29.97 tbn, 29.97 tbc
Successfully opened the file.
Parsing a group of options: output file -.
Applying option vf (set video filters) with argument decimate.
Applying option f (force format) with argument null.
Successfully parsed a group of options.
Opening an output file: -.
Successfully opened the file.
detected 8 logical cores
[graph 0 input from stream 0:0 @ 0x32c1580] Setting 'video_size' to value '1920x1080'
[graph 0 input from stream 0:0 @ 0x32c1580] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0x32c1580] Setting 'time_base' to value '1001/30000'
[graph 0 input from stream 0:0 @ 0x32c1580] Setting 'pixel_aspect' to value '0/1'
[graph 0 input from stream 0:0 @ 0x32c1580] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0x32c1580] Setting 'frame_rate' to value '30000/1001'
[graph 0 input from stream 0:0 @ 0x32c1580] w:1920 h:1080 pixfmt:yuv420p tb:1001/30000 fr:30000/1001 sar:0/1 sws_param:flags=2
[AVFilterGraph @ 0x32c0280] query_formats: 3 queried, 2 merged, 0 already done, 0 delayed
[Parsed_decimate_0 @ 0x32c1160] FPS: 30000/1001 -> 24000/1001
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf57.21.100
    Stream #0:0, 0, 1001/24000: Video: wrapped_avframe, 1 reference frame, yuv420p, 1920x1080, 1001/24000, q=2-31, 200 kb/s, 23.98 fps, 23.98 tbn, 23.98 tbc
    Metadata:
      encoder         : Lavc57.21.100 wrapped_avframe
Stream mapping:
  Stream #0:0 -> #0:0 (ffv1 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
    Last message repeated 11 times
[Parsed_decimate_0 @ 0x32c1160] 1/5 frame drop:
[Parsed_decimate_0 @ 0x32c1160]   #1: totdiff=7fffffffffffffff maxbdiff=7fffffffffffffff sc [DROP]
[Parsed_decimate_0 @ 0x32c1160]   #2: totdiff=00ca660c maxbdiff=000130d8
[Parsed_decimate_0 @ 0x32c1160]   #3: totdiff=015715f3 maxbdiff=00022e12
[Parsed_decimate_0 @ 0x32c1160]   #4: totdiff=00c6e459 maxbdiff=0001df5b
[Parsed_decimate_0 @ 0x32c1160]   #5: totdiff=000d4fce maxbdiff=00000d05 lowest
[Parsed_decimate_0 @ 0x32c1160] 1/5 frame drop:
[Parsed_decimate_0 @ 0x32c1160]   #1: totdiff=0093eda5 maxbdiff=0001b481
[Parsed_decimate_0 @ 0x32c1160]   #2: totdiff=0089d968 maxbdiff=00019ebc
[Parsed_decimate_0 @ 0x32c1160]   #3: totdiff=0080a999 maxbdiff=00017223
[Parsed_decimate_0 @ 0x32c1160]   #4: totdiff=007c98d0 maxbdiff=00018261
[Parsed_decimate_0 @ 0x32c1160]   #5: totdiff=00099bf0 maxbdiff=00000f6b lowest [DROP]
[Parsed_decimate_0 @ 0x32c1160] 1/5 frame drop:
[Parsed_decimate_0 @ 0x32c1160]   #1: totdiff=0079855c maxbdiff=0001ab54
[Parsed_decimate_0 @ 0x32c1160]   #2: totdiff=008401c0 maxbdiff=0001bf8e
[Parsed_decimate_0 @ 0x32c1160]   #3: totdiff=007db8bb maxbdiff=0001ade7
[Parsed_decimate_0 @ 0x32c1160]   #4: totdiff=007f553c maxbdiff=0001806d
[Parsed_decimate_0 @ 0x32c1160]   #5: totdiff=0010b14d maxbdiff=00001d9a lowest [DROP]
[Parsed_decimate_0 @ 0x32c1160] 1/5 frame drop:
[Parsed_decimate_0 @ 0x32c1160]   #1: totdiff=007b5d19 maxbdiff=0001691f
[Parsed_decimate_0 @ 0x32c1160]   #2: totdiff=0076c12d maxbdiff=00012a61
[Parsed_decimate_0 @ 0x32c1160]   #3: totdiff=006cc5ab maxbdiff=00014404
[Parsed_decimate_0 @ 0x32c1160]   #4: totdiff=005b2dca maxbdiff=0000e7ce
[Parsed_decimate_0 @ 0x32c1160]   #5: totdiff=000fe7bc maxbdiff=0000150d lowest [DROP]
[Parsed_decimate_0 @ 0x32c1160] 1/5 frame drop:
[output stream 0:0 @ 0x32c17c0] EOF on sink link output stream 0:0:default.
No more output streams to write to, finishing.
frame=   16 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.70 bitrate=N/A speed=2.36x
video:8kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Input file #0 (fieldmatched.avi):
  Input stream #0:0 (video): 20 packets read (10258995 bytes); 20 frames decoded;
  Total: 20 packets (10258995 bytes) demuxed
Output file #0 (pipe:):
  Output stream #0:0 (video): 16 frames encoded; 16 packets muxed (7680 bytes);
  Total: 16 packets (7680 bytes) muxed
20 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0x32c4f20] Statistics: 10325678 bytes read, 4 seeks

The first frame is incorrectly dropped.
Samples in http://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket4990/
Patch sent:
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/206462

comment:3 Changed 2 years ago by richardpl

  • Resolution set to fixed
  • Status changed from open to closed

Fixed in 962727acb4e3c4dff856b48460ddcf747fcfda93.
Tested with mpv vapoursynth filter, and this script:

import vapoursynth as vs
core = vs.get_core()
video = video_in
video = core.vivtc.VDecimate(video)
video.set_output()

comment:4 Changed 2 years ago by cehoyos

  • Resolution fixed deleted
  • Status changed from closed to reopened

I still see the same issue as originally (badly explained here because I only understood later on), or perhaps originally after Nicolas' patch which is why I didn't close this ticket:
Instead of dropping either the fourth or fifth frame (that are visually identical) of http://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket4990/fieldmatched.avi, the first frame is dropped (because this is the default if no matching frames can be found).
The question is now: Does the same happen with AVS (in which case this ticket is invalid, not fixed) or does AVS drop one of the visually identical frames (in which case this is a valid ticket)?

comment:5 Changed 2 years ago by cehoyos

In the first block of five frames, the fifth frame should be dropped, but the first frame is dropped instead:

$ ffmpeg -loglevel 50 -i fieldmatched.avi -vf decimate -f null -
ffmpeg version N-81500-ga648391 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      55. 29.100 / 55. 29.100
  libavcodec     57. 54.101 / 57. 54.101
  libavformat    57. 48.101 / 57. 48.101
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 58.100 /  6. 58.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '50'.
Reading option '-i' ... matched as input file with argument 'fieldmatched.avi'.
Reading option '-vf' ... matched as option 'vf' (set video filters) with argument 'decimate'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'null'.
Reading option '-' ... matched as output file.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument 50.
Successfully parsed a group of options.
Parsing a group of options: input file fieldmatched.avi.
Successfully parsed a group of options.
Opening an input file: fieldmatched.avi.
[file @ 0x1fcbd20] Setting default whitelist 'file,crypto'
[avi @ 0x1fcb400] Format avi probed with size=2048 and score=100
[avi @ 0x1fcbda0] use odml:1
[avi @ 0x1fcb400] Before avformat_find_stream_info() pos: 5720 bytes read:98952 seeks:4 nb_streams:1
[avi @ 0x1fcb400] parser not found for codec ffv1, packets or times may be invalid.
    Last message repeated 1 times
[avi @ 0x1fcb400] All info found
[avi @ 0x1fcb400] After avformat_find_stream_info() pos: 635993 bytes read:696457 seeks:4 frames:1
Input #0, avi, from 'fieldmatched.avi':
  Metadata:
    encoder         : Lavf57.20.100
  Duration: 00:00:00.67, start: 0.000000, bitrate: 123059 kb/s
    Stream #0:0, 1, 1001/30000: Video: ffv1, 1 reference frame (FFV1 / 0x31564646), yuv420p, 1920x1080, 0/1, 129457 kb/s, 29.97 fps, 29.97 tbr, 29.97 tbn, 29.97 tbc
Successfully opened the file.
Parsing a group of options: output file -.
Applying option vf (set video filters) with argument decimate.
Applying option f (force format) with argument null.
Successfully parsed a group of options.
Opening an output file: -.
Successfully opened the file.
detected 8 logical cores
[graph 0 input from stream 0:0 @ 0x1fd0060] Setting 'video_size' to value '1920x1080'
[graph 0 input from stream 0:0 @ 0x1fd0060] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0x1fd0060] Setting 'time_base' to value '1001/30000'
[graph 0 input from stream 0:0 @ 0x1fd0060] Setting 'pixel_aspect' to value '0/1'
[graph 0 input from stream 0:0 @ 0x1fd0060] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0x1fd0060] Setting 'frame_rate' to value '30000/1001'
[graph 0 input from stream 0:0 @ 0x1fd0060] w:1920 h:1080 pixfmt:yuv420p tb:1001/30000 fr:30000/1001 sar:0/1 sws_param:flags=2
[AVFilterGraph @ 0x1fd0220] query_formats: 3 queried, 2 merged, 0 already done, 0 delayed
[Parsed_decimate_0 @ 0x1fd1100] FPS: 30000/1001 -> 24000/1001
[null @ 0x1fce1c0] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf57.48.101
    Stream #0:0, 0, 1001/24000: Video: wrapped_avframe, 1 reference frame, yuv420p, 1920x1080, 0/1, q=2-31, 200 kb/s, 23.98 fps, 23.98 tbn, 23.98 tbc
    Metadata:
      encoder         : Lavc57.54.101 wrapped_avframe
Stream mapping:
  Stream #0:0 -> #0:0 (ffv1 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
    Last message repeated 11 times
[Parsed_decimate_0 @ 0x1fd1100] 1/5 frame drop:
[Parsed_decimate_0 @ 0x1fd1100]   #1: totdiff=7fffffffffffffff maxbdiff=7fffffffffffffff sc [DROP]
[Parsed_decimate_0 @ 0x1fd1100]   #2: totdiff=00ca660c maxbdiff=000130d8
[Parsed_decimate_0 @ 0x1fd1100]   #3: totdiff=015715f3 maxbdiff=00022e12
[Parsed_decimate_0 @ 0x1fd1100]   #4: totdiff=00c6e459 maxbdiff=0001df5b
[Parsed_decimate_0 @ 0x1fd1100]   #5: totdiff=000d4fce maxbdiff=00000d05 lowest
[Parsed_decimate_0 @ 0x1fd1100] 1/5 frame drop:
[Parsed_decimate_0 @ 0x1fd1100]   #1: totdiff=0093eda5 maxbdiff=0001b481
[Parsed_decimate_0 @ 0x1fd1100]   #2: totdiff=0089d968 maxbdiff=00019ebc
[Parsed_decimate_0 @ 0x1fd1100]   #3: totdiff=0080a999 maxbdiff=00017223
[Parsed_decimate_0 @ 0x1fd1100]   #4: totdiff=007c98d0 maxbdiff=00018261
[Parsed_decimate_0 @ 0x1fd1100]   #5: totdiff=00099bf0 maxbdiff=00000f6b lowest [DROP]
[Parsed_decimate_0 @ 0x1fd1100] 1/5 frame drop:
[Parsed_decimate_0 @ 0x1fd1100]   #1: totdiff=0079855c maxbdiff=0001ab54
[Parsed_decimate_0 @ 0x1fd1100]   #2: totdiff=008401c0 maxbdiff=0001bf8e
[Parsed_decimate_0 @ 0x1fd1100]   #3: totdiff=007db8bb maxbdiff=0001ade7
[Parsed_decimate_0 @ 0x1fd1100]   #4: totdiff=007f553c maxbdiff=0001806d
[Parsed_decimate_0 @ 0x1fd1100]   #5: totdiff=0010b14d maxbdiff=00001d9a lowest [DROP]
[Parsed_decimate_0 @ 0x1fd1100] 1/5 frame drop:
[Parsed_decimate_0 @ 0x1fd1100]   #1: totdiff=007b5d19 maxbdiff=0001691f
[Parsed_decimate_0 @ 0x1fd1100]   #2: totdiff=0076c12d maxbdiff=00012a61
[Parsed_decimate_0 @ 0x1fd1100]   #3: totdiff=006cc5ab maxbdiff=00014404
[Parsed_decimate_0 @ 0x1fd1100]   #4: totdiff=005b2dca maxbdiff=0000e7ce
[Parsed_decimate_0 @ 0x1fd1100]   #5: totdiff=000fe7bc maxbdiff=0000150d lowest [DROP]
[Parsed_decimate_0 @ 0x1fd1100] 1/5 frame drop:
[output stream 0:0 @ 0x1fd1d20] EOF on sink link output stream 0:0:default.
No more output streams to write to, finishing.
frame=   16 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.70 bitrate=N/A speed=2.44x
video:8kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Input file #0 (fieldmatched.avi):
  Input stream #0:0 (video): 20 packets read (10258995 bytes); 20 frames decoded;
  Total: 20 packets (10258995 bytes) demuxed
Output file #0 (pipe:):
  Output stream #0:0 (video): 16 frames encoded; 16 packets muxed (7808 bytes);
  Total: 16 packets (7808 bytes) muxed
20 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0x1fd4040] Statistics: 10325678 bytes read, 4 seeks

comment:6 Changed 2 years ago by cehoyos

I remember that I wondered if the calculation for dupthresh (and scthresh) scales correctly: The deeper issue with the sample video here is that no duplicate is ever found ("dup" would be printed in above output).

comment:7 Changed 2 years ago by richardpl

  • Resolution set to invalid
  • Status changed from reopened to closed

Please, filter behaves as original vapoursynth filter and it is based on that one not avisynth one, there is nothing to fix.

comment:8 Changed 2 years ago by cehoyos

But doesn't this indicate that there is still a bug in the filter: The fourth and fifth frame are very similar imo.
Can you confirm that the dupthresh calculation is correct?

comment:9 Changed 2 years ago by cehoyos

Works with blocky=64 and chroma=0 (and dupthresh=1.3).

Note: See TracTickets for help on using tickets.