Opened 5 years ago
Last modified 5 weeks ago
#8063 open defect
`-vsync cfr` tends to duplicate the wrong frames
Reported by: | gdgsdg123 | Owned by: | |
---|---|---|---|
Priority: | normal | Component: | tools |
Version: | git-master | Keywords: | fftools vsync fps_mode |
Cc: | MasterQuestionable | Blocked By: | |
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
With a VFR input, -vsync cfr
may alter the playback experience by duplicating the wrong frames. (appeared to be... exactly 1 frame later)
Simple illustration of the problem:
(each number represents a frame of unique content)
-vsync vfr
(expressed in CFR manner):
1 1 2 2 3
-vsync cfr
:
1 2 2 3 3
Note:
The above happened in MP4, MKV container, failed to reproduce (seemed working correctly) in AVI container. (tested x264, and appeared to be codec dependent... as with magicyuv it happens in AVI also, and more severe: broken frames (part of the duplicated frames have incorrect color))
Attachments (5)
Change History (18)
comment:1 by , 5 years ago
Resolution: | → needs_more_info |
---|---|
Status: | new → closed |
by , 5 years ago
comment:2 by , 5 years ago
follow-up: 4 comment:3 by , 5 years ago
vsync takes effect before encoding and muxing so choice of encoder and muxer is irrelevant - one caveat being that if vsync is not specified by the user ffmpeg will set it as per muxer flags.
comment:4 by , 5 years ago
Resolution: | needs_more_info |
---|---|
Status: | closed → reopened |
Replying to Gyan:
one caveat being that if vsync is not specified by the user ffmpeg will set it as per muxer flags.
In another word: always explicitly set the preferred vsync
method, unless you want FFmpeg to choose one randomly for you...
comment:5 by , 5 years ago
It's not random. It's chosen as per muxer flags which are fixed in code and rarely changed.
follow-up: 8 comment:7 by , 5 years ago
Component: | undetermined → ffmpeg |
---|---|
Keywords: | fps added |
While the description is highly misleading, I believe that a (minor) issue can be reproduced with the sample file.
follow-up: 10 comment:9 by , 5 years ago
comment:10 by , 5 years ago
Replying to cehoyos:
Feel free to provide such clips.
I have already... If you had the interest to download:
https://www.youtube.com/playlist?list=PLXvoB-d-4W1kKvr3CA20DrcU1HzTXibAX (check the descriptions)
comment:11 by , 5 weeks ago
I also encountered this issue. I've attached a small 6 frame test case named Null-Frame-Example.avi. I've also attached screenshots of how the null frame is supposed to look, and how it looks after FFmpeg has incorrectly handled it.
This test case is video that was initially recorded from a capture device using VirtualDub 1.9, and then trimmed using VirtualDub2 using Direct Stream Copy mode (i.e. no recompression).
During capture, VirtualDub inserted what are apparently known as "null frames" or "drop frames" in the AVI file.
Here is a table of frames in the attached test case.
Frame number | Frame type |
0 | Key frame |
1 | Key frame |
2 | Key frame |
3 | Null/drop frame |
4 | Key frame |
5 | Key frame |
If I visually inspect the input file in VirtualDub2, 6 total frames are shown. Viewing frame index 3 will show the frame data from frame 2, and is the output I would also expect from FFmpeg. It is denoted with the special [D] tag in the status bar, indicating it is a null/drop frame. That is, a null frame is effectively a way of repeating the previous frame. And, that is how VirtualDub inserts placeholder frames during capture when the frames from the capture device are not keeping up with the clock.
In other words, this is the expected visual output of FFmpeg, and the observed frame table in VirtualDub2 when viewing the input:
Frame number | Frame data |
0 | Unique |
1 | Unique |
2 | Unique |
3 | Duplicate of frame 2 |
4 | Unique |
5 | Unique |
Now, suppose I run the following command:
# NOTE: using "-vsync cfr" will yield the same result ffmpeg -i Null-Frame-Example.avi -fps_mode cfr -c:a flac -c:v ffv1 Null-Frame-Example.mkv
I then open this output MKV file, and observe the following actual / erroneous output:
Frame number | Frame data |
0 | Unique |
1 | Unique |
2 | Unique |
3 | Duplicate of frame 4 |
4 | Unique |
5 | Unique |
This is not correct; the frame index 3 should be from frame 2, as seen in the earlier table.
(Side note: I did not know that AVI files could even have these kinds of frames (most end users probably don't). My first attempt without any "vsync" or "fps_mode" parameter at all led to totally broken frames in the output file: the MKV file still has 6 frames according to VirtualDub2, but it gives a "frame missing" error when trying to view frame 3. It might be worth a consideration to make the default settings do the right thing somehow / always duplicate the (correct) frame when encountering null/drop frames.)
(How I got here: AviSynth's FFmpeg-based sources would silently drop these null frames, leading to audio/video sync issues in the output file!)
by , 5 weeks ago
Attachment: | Null-Frame-Example.avi added |
---|
by , 5 weeks ago
Attachment: | Expected output - Null-Frame-Example AVI viewed in VirtualDub.png added |
---|
by , 5 weeks ago
Attachment: | Actual output - Null-Frame-Example output MKV viewed in VirtualDub.png added |
---|
by , 5 weeks ago
Attachment: | 123.avi.xml added |
---|
͏ https://trac.ffmpeg.org/raw-attachment/ticket/8063/123.avi
͏ (~ 24.09 KiB; AVI: H.264 (AVC) video, 5 s, 1 FPS VFR (avg: 0.6, min: 0.5, max: 1), 800x600, RGB, ~ 18.42 KiB; ... ~ 5.67 KiB)
͏ ffprobe -v warning -hide_banner -threads 0 -show_entries "frame" -select_streams v:0 -of "xml" "123.avi" -o "123.avi.xml"
͏ Note:
͏ The interpreted "duration" alike appeared wrong.
͏ Would it be the cause? ("-vsync" misused which?)
comment:12 by , 5 weeks ago
Cc: | added |
---|---|
Component: | ffmpeg → tools |
Keywords: | fftools vsync fps_mode added; fps removed |
Status: | reopened → open |
͏ I believe ͏"123.avi" should be the most straight-forward sample.
͏ See [ https://trac.ffmpeg.org/ticket/10928 ] for the Trac bug...
comment:13 by , 5 weeks ago
͏ Seemingly container dependent... probably for alike "hidden secrets" in comment:3?
͏ See also: https://ffmpeg.org/ffmpeg.html#Advanced-options
͏ "and more severe: broken frames (part of the duplicated frames have incorrect color)"
͏ Looks like bad Inter-compression handling. Codec bug maybe?
͏ Or wrong frame sent to codec per the problem..?
͏ Alike 1-frame-off issues may be serious:
͏ https://trac.ffmpeg.org/ticket/11060
Replying to gdgsdg123:
Managed to reproduce it in AVI container:
Frame information of 123.avi:
Expected frame information of cfr.avi: