Opened 5 years ago

Closed 3 years ago

Last modified 23 months ago

#7927 closed defect (invalid)

Video copy from mkv to mp4 causes variable frame rate from constant frame rate

Reported by: Mads Johansen Owned by:
Priority: normal Component: undetermined
Version: git-master Keywords:
Cc: RytoEX@gmail.com Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

I am trying to convert from MKV to MP4 or MOV with a constant frame rate and the resulting file has a variable frame rate.
In running ffmpeg -i FILE.mkv -c:v copy FILE.mp4, I expect a pure container switch, and that the frame rate to stay exactly the same. The reverse (mp4 to mkv) causes no frame rate change.

In mediainfo for source:
Frame rate mode : Constant
Frame rate : 29.970 (30000/1001) FPS

In mediainfo for destination:
Frame rate mode : Variable
Frame rate : 31.946 FPS
Minimum frame rate : 27.778 FPS
Maximum frame rate : 16 000.000 FPS
Original frame rate : 29.970 (30000/1001) FPS

The command I use is: ffmpeg.exe -i Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mkv -c:v copy -an Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mp4

I have a 2.63 MB mkv file for reproduction available, output of video copy is as below:

ffmpeg -v 9 -loglevel 99 -i Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mkv -c:v copy -an Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mp4
ffmpeg version N-93913-g1a74b04737 Copyright (c) 2000-2019 the FFmpeg developers

built with gcc 8.3.0 (Rev2, Built by MSYS2 project)
configuration: --disable-autodetect --enable-amf --enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-nvenc --enable-zlib --enable-sdl2 --disable-debug --enable-ffnvcodec --enable-nvdec --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libdav1d --enable-fontconfig --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libwavpack --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --enable-libxvid --enable-libaom --enable-libopenmpt --enable-version3 --enable-mbedtls --extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++ --extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv
libavutil 56. 28.100 / 56. 28.100
libavcodec 58. 52.102 / 58. 52.102
libavformat 58. 27.103 / 58. 27.103
libavdevice 58. 7.100 / 58. 7.100
libavfilter 7. 54.101 / 7. 54.101
libswscale 5. 4.101 / 5. 4.101
libswresample 3. 4.100 / 3. 4.100
libpostproc 55. 4.100 / 55. 4.100

Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-i' ... matched as input url with argument 'Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mkv'.
Reading option '-c:v' ... matched as option 'c' (codec name) with argument 'copy'.
Reading option '-an' ... matched as option 'an' (disable audio) with argument '1'.
Reading option 'Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mp4' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument 9.
Successfully parsed a group of options.
Parsing a group of options: input url Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mkv.
Successfully parsed a group of options.
Opening an input file: Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mkv.
[NULL @ 000001dd98e32b40] Opening 'Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mkv' for reading
[file @ 000001dd98e33c40] Setting default whitelist 'file,crypto'
Probing matroska,webm score:100 size:2048
[matroska,webm @ 000001dd98e32b40] Format matroska,webm probed with size=2048 and score=100
st:0 removing common factor 1000000 from timebase
st:1 removing common factor 1000000 from timebase
[matroska,webm @ 000001dd98e32b40] Before avformat_find_stream_info() pos: 4934 bytes read:32768 seeks:0 nb_streams:2
[h264 @ 000001dd98e36800] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 000001dd98e36800] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 000001dd98e36800] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 000001dd98e36800] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 000001dd98e36800] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 000001dd98e36800] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 000001dd98e36800] Format yuvj420p chosen by get_format().
[h264 @ 000001dd98e36800] Reinit context to 1920x1088, pix_fmt: yuvj420p
[h264 @ 000001dd98e36800] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 000001dd98e36800] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
[h264 @ 000001dd98e36800] no picture ooo
[h264 @ 000001dd98e36800] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 000001dd98e36800] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
[h264 @ 000001dd98e36800] no picture ooo
[h264 @ 000001dd98e36800] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 000001dd98e36800] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[h264 @ 000001dd98e36800] no picture
[h264 @ 000001dd98e36800] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 000001dd98e36800] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
[h264 @ 000001dd98e36800] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 000001dd98e36800] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
[h264 @ 000001dd98e36800] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 000001dd98e36800] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[h264 @ 000001dd98e36800] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 000001dd98e36800] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
[h264 @ 000001dd98e36800] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 000001dd98e36800] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
[h264 @ 000001dd98e36800] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 000001dd98e36800] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[matroska,webm @ 000001dd98e32b40] All info found
[matroska,webm @ 000001dd98e32b40] stream 0: start_time: 0.067 duration: -9223372036854776.000
[matroska,webm @ 000001dd98e32b40] stream 1: start_time: 0.000 duration: -9223372036854776.000
[matroska,webm @ 000001dd98e32b40] format: start_time: 0.000 duration: 1.104 bitrate=20018 kb/s
[matroska,webm @ 000001dd98e32b40] After avformat_find_stream_info() pos: 1040235 bytes read:1057837 seeks:0 frames:28
Input #0, matroska,webm, from 'Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mkv':

Metadata:

COMPATIBLE_BRANDS: qt niko
MAJOR_BRAND : qt
MINOR_VERSION : 537331968
ENCODER : Lavf58.27.103

Duration: 00:00:01.10, start: 0.000000, bitrate: 20018 kb/s

Stream #0:0(eng), 11, 1/1000: Video: h264 (High), 1 reference frame, yuvj420p(pc, smpte170m/bt709/bt470m, progressive, left), 1920x1080 (1920x1088), 0/1, 29.97 fps, 29.97 tbr, 1k tbn, 59.94 tbc (default)
Metadata:

DURATION : 00:00:01.104000000

Stream #0:1(eng), 17, 1/1000: Audio: vorbis, 48000 Hz, stereo, fltp (default)
Metadata:

ENCODER : Lavc58.52.102 libvorbis
DURATION : 00:00:01.003000000

Successfully opened the file.
Parsing a group of options: output url Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mp4.
Applying option c:v (codec name) with argument copy.
Applying option an (disable audio) with argument 1.
Successfully parsed a group of options.
Opening an output file: Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mp4.
[file @ 000001dd99b6b040] Setting default whitelist 'file,crypto'
Successfully opened the file.
Output #0, mp4, to 'Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mp4':

Metadata:

COMPATIBLE_BRANDS: qt niko
MAJOR_BRAND : qt
MINOR_VERSION : 537331968
encoder : Lavf58.27.103
Stream #0:0(eng), 0, 1/16000: Video: h264 (High), 1 reference frame (avc1 / 0x31637661), yuvj420p(pc, smpte170m/bt709/bt470m, progressive, left), 1920x1080 (0x0), 0/1, q=2-31, 29.97 fps, 29.97 tbr, 16k tbn, 1k tbc (default)
Metadata:

DURATION : 00:00:01.104000000

Stream mapping:

Stream #0:0 -> #0:0 (copy)

Press [q] to stop, ? for help
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[mp4 @ 000001dd98f72880] Non-monotonous DTS in output stream 0:0; previous: 538, current: 48; changing to 539. This may result in incorrect timestamps in the output file.
[matroska,webm @ 000001dd98e32b40] Element at 0x2a26fb ending at 0x2a2739 exceeds containing master element ending at 0x2a26f6
No more output streams to write to, finishing.
frame= 31 fps=0.0 q=-1.0 Lsize= 2680kB time=00:00:00.97 bitrate=22608.3kbits/s speed=13.6x
video:2679kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.049911%
Input file #0 (Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mkv):

Input stream #0:0 (video): 31 packets read (2742890 bytes);
Input stream #0:1 (audio): 17 packets read (4456 bytes);
Total: 48 packets (2747346 bytes) demuxed

Output file #0 (Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mp4):

Output stream #0:0 (video): 31 packets muxed (2742890 bytes);
Total: 31 packets (2742890 bytes) muxed

0 frames successfully decoded, 0 decoding errors
[AVIOContext @ 000001dd9911e980] Statistics: 2 seeks, 14 writeouts
[AVIOContext @ 000001dd98e3bec0] Statistics: 2762553 bytes read, 0 seeks

Change History (22)

comment:1 by Mads Johansen, 5 years ago

Last edited 3 years ago by Mads Johansen (previous) (diff)

comment:2 by mkver, 5 years ago

The Matroska container doesn't store DTS (decoding timestamps), only PTS (presentation timestamps); but mp4 stores (i.e. requires) both DTS and PTS, so FFmpeg needs to guess the DTS. This guess works well when the amount of reorderes packets is known and there is no open GOP at the beginning. Your file has an open GOP at the beginning. Notice how the two frames from track 1 (the video track) that follow the first frame have a lower timestamp than the first frame:

I frame, track 1, timestamp 00:00:00.067000000, size 271247, adler 0x89f7e858
I frame, track 2, timestamp 00:00:00.000000000, size 79, adler 0xcf102aef
P frame, track 1, timestamp 00:00:00.003000000, size 77940, adler 0xa16fe9fe
I frame, track 2, timestamp 00:00:00.003000000, size 71, adler 0x6f6b24b3
I frame, track 2, timestamp 00:00:00.006000000, size 284, adler 0xc7e38d66
I frame, track 2, timestamp 00:00:00.018000000, size 276, adler 0x5d1c87cb
P frame, track 1, timestamp 00:00:00.036000000, size 46893, adler 0xb38d083c
I frame, track 2, timestamp 00:00:00.039000000, size 286, adler 0x728f8c1f
I frame, track 2, timestamp 00:00:00.060000000, size 298, adler 0xaa5f9d36
P frame, track 1, timestamp 00:00:00.170000000, size 150864, adler 0xc434e645

I wouldn't be surprised if there were already a ticket for exactly this issue.

PS: FFmpeg also guesses DTS when it e.g. remuxes to Matroska; so the conversion mkv->mkv suffers from the same issue.

comment:3 by Mads Johansen, 5 years ago

Thank you for the reply.

So the solution is to create DTS? If so is there a way to force DTS calculations via the command line?

comment:4 by mkver, 5 years ago

FFmpeg already tries to create DTS -- and fails, because of the open GOP.

comment:5 by Mads Johansen, 5 years ago

Resolution: worksforme
Status: newclosed

The solution I found for getting a constant frame rate was to push the video through virtualdub2, selecting x264 10 bit, 4:2:0 and CRF 1.
This gives lossless conversion according to https://slhck.info/video/2017/02/24/crf-guide.html and https://trac.ffmpeg.org/wiki/Encode/H.264

comment:6 by Carl Eugen Hoyos, 3 years ago

Resolution: worksformeinvalid

There may be an issue but the sample is not ideal afaict.

comment:7 by RytoEX, 3 years ago

Cc: RytoEX@gmail.com added

What would be an ideal sample? I'm willing to provide sample data for this.

comment:8 by Carl Eugen Hoyos, 3 years ago

One that was not created with FFmpeg, so that the issue that you are trying to show (I am not completely convinced that it exists) is not already reproducible with the input file.
Additionally, please try not to use MediaInfo to show an issue with FFmpeg: This should not be necessary if there is an issue.

Last edited 3 years ago by Carl Eugen Hoyos (previous) (diff)

in reply to:  7 ; comment:9 by Balling, 3 years ago

Replying to RytoEX:

What would be an ideal sample? I'm willing to provide sample data for this.

But this is the point. You original file is recognized as VFR! So if there is a bug it is in the initial parser of the original file. And that is due to it 30/1.001 fps. If you used 25.000 it will not be a problem.

ffmpeg -i Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mkv -vf vfrdet -an -f null -

Also, wanted to ask. Is you file really transfer ITU-R BT.470-6 System M??

Last edited 3 years ago by Balling (previous) (diff)

in reply to:  9 ; comment:10 by Mads Johansen, 3 years ago

Resolution: invalid
Status: closedreopened

Replying to Balling:

Replying to RytoEX:

What would be an ideal sample? I'm willing to provide sample data for this.

But this is the point. You original file is recognized as VFR! So if there is a bug it is in the initial parser of the original file. And that is due to it 30/1.001 fps. If you used 25.000 it will not be a problem.

ffmpeg -i Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mkv -vf vfrdet -an -f null -

Also, wanted to ask. Is you file really transfer ITU-R BT.470-6 System M??

I'm the original author :) And yes, that's what a Nikon D600 produces. Why? No clue.

in reply to:  10 comment:11 by Balling, 3 years ago

Replying to madsfoto:

I'm the original author :) And yes, that's what a Nikon D600 produces. Why? No clue.

Because ITU-R BT.470-6 System M is ideal 2.2000 gamma, I suppose. While sRGB is piece wise with 2.4 that gives almost 2.2 and BT.709 is more like 1.9 gamma with needed BT.1886 correction. Even Adobe RGB 1998 transfer is 563/256, or 2.19921875, not perfect 2.2.

Is Nikon really using mkv? mp4 does not have such a problem whatsover.

Last edited 3 years ago by Balling (previous) (diff)

comment:12 by Carl Eugen Hoyos, 3 years ago

Resolution: invalid
Status: reopenedclosed

Please do not reopen this ticket unless you can provide an input file that was not created by FFmpeg.

in reply to:  8 ; comment:13 by RytoEX, 3 years ago

Replying to cehoyos:

One that was not created with FFmpeg, so that the issue that you are trying to show (I am not completely convinced that it exists) is not already reproducible with the input file.
Additionally, please try not to use MediaInfo to show an issue with FFmpeg: This should not be necessary if there is an issue.

Replying to cehoyos:

Please do not reopen this ticket unless you can provide an input file that was not created by FFmpeg.

So you need an MKV file created by something that is not FFmpeg, and does not rely on FFmpeg code? Why would an MKV file created by FFmpeg not suffice?

I'm discussing with others on how best to produce ideal samples for you that meet your criteria.

in reply to:  13 comment:14 by Balling, 3 years ago

Replying to RytoEX:

Please note that Nikon can also use ffmpeg internally. D:)

Last edited 3 years ago by Balling (previous) (diff)

comment:15 by Balling, 3 years ago

2 questions here: what is the original file produced by Nikon D600? Please give it or if it the mp4 file, just say so.

mkver, can you maybe say 100% whether this is a bug or not? Also I suppose the fix here is to use annex B hack, an I right?

in reply to:  9 comment:16 by dojima, 3 years ago

Is anyone able to take any MKV file and copy it to an MP4 container with ffmpeg and have it remain at a constant framerate? I'm no expert, so I can't begin to guess why this doesn't work intuitively. The only way I am able to do this is by using -video_track_timescale, but it doesn't work with 59.94 or presumably 29.97, 23.976, etc. In order to convert a 59.94 constant MKV to MP4, I had to output it as a raw stream with mkvextract and then put that into an MP4 container with MP4Box.

Replying to Balling:

But this is the point. You original file is recognized as VFR! So if there is a bug it is in the initial parser of the original file. And that is due to it 30/1.001 fps. If you used 25.000 it will not be a problem.

ffmpeg -i Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mkv -vf vfrdet -an -f null -

Near as I can tell, any MKV run with that command will show VFR. Is that intended behavior?

comment:17 by Balling, 3 years ago

There is a special thing that should be done for AVC.

ffmpeg -i vfr.mkv -c copy -bsf:v "h264_metadata=tick_rate=60000/1001:fixed_frame_rate_flag=1" -video_track_timescale 60000 "output.mp4"

fixed_frame_rate_flag is creating a special flag is SPS of bitstream itself. Now, that still does not work! Now, apparently it a very common problem. https://github.com/slhck/ffmpeg-normalize/issues/95

Last edited 3 years ago by Balling (previous) (diff)

in reply to:  8 ; comment:18 by dojima, 3 years ago

Replying to Carl Eugen Hoyos:

One that was not created with FFmpeg, so that the issue that you are trying to show (I am not completely convinced that it exists) is not already reproducible with the input file.

Can you be a little clearer about what you mean by this? I'd like to help resolve this issue since it's annoying for me to work around, but I'm unsure what you need. You can take literally take any CFR video file and copy it into MKV then MP4 to see this issue clearly.

Here's a CFR MKV that ffmpeg will change to VFR after remuxing to MP4 unless -video_track_timescale 60 is used.

https://github.com/haasn/interpolation-samples/raw/master/60.000.mkv

in reply to:  18 comment:19 by Balling, 3 years ago

Replying to dojima:

https://github.com/haasn/interpolation-samples/raw/master/60.000.mkv

That video is interger frame rate 60.000 not 60/1.001 which is still dominating frame rate. Just note that.

My command is from here. https://forum.doom9.org/showthread.php?p=1876241

Last edited 3 years ago by Balling (previous) (diff)

comment:20 by dojima, 3 years ago

Yeah, but regardless of the framerate, I wouldn't expect it to change to VFR with simple container manipulations.

comment:22 by Balling, 23 months ago

Apprently even doing ffmpeg.exe -i Convert_to_mp4_ConstantFramerate_to_VariableFramerate.mkv -c copy cnas.h264

ffmpeg.exe -r 29.97002997 -i cnas.h264 -c copy fcwejsdvfw.mp4

will not fully fix it, since

ffmpeg.exe -i fcwejsdvfw.mp4 -vf vfrdet -f null -

still says

[Parsed_vfrdet_0 @ 000002aef1294540] VFR:0.037037 (1/26) min: 0 max: 1001 avg: 0

because one of

[null @ 0000024a1bbceac0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 29 >= 29

Last edited 23 months ago by Balling (previous) (diff)
Note: See TracTickets for help on using tickets.