Invalid pts and duration generated for mp4 when converting from webm

[jyavenard]

Summary of the bug:
How to reproduce:

% ffmpeg -i MozilleTest -c:v libx264 test.mp4

ffmpeg version 4.02
built on macos 10.13.6 (from brew)

Running ffprobe on the output:
ffprobe -show_entries packet=pts_time,duration_time,stream_index ~/Downloads/test.mp4

You can see that all frames have been marked with a duration of 1ms only.

[PACKET]
stream_index=0
pts_time=0.000000
duration_time=0.001000
[/PACKET]
[PACKET]
stream_index=0
pts_time=0.004000
duration_time=0.001000
[/PACKET]
[PACKET]
stream_index=0
pts_time=0.002000
duration_time=0.001000
[/PACKET]
[PACKET]
stream_index=0
pts_time=0.001000
duration_time=0.001000
[/PACKET]
[PACKET]
stream_index=0
pts_time=0.003000
duration_time=0.001000
[/PACKET]
[PACKET]
stream_index=0
pts_time=0.008000
duration_time=0.001000
[/PACKET]
[PACKET]

It actually looks like something weird is occurring with the webm demuxer/remuxer

as simply remuxing the webm content into another:

ffmpeg -i MozillaTest -c:v copy -c:a copy -t 10 test.webm

Also produce a file where all video frames are marked as being 1ms long:

[PACKET]
stream_index=0
pts_time=0.001000
duration_time=0.001000
[/PACKET]
[PACKET]
stream_index=0
pts_time=0.036000
duration_time=0.001000
[/PACKET]
[PACKET]
stream_index=0
pts_time=0.065000
duration_time=0.001000
[/PACKET]
[PACKET]
stream_index=0
pts_time=0.097000
duration_time=0.001000
[/PACKET]
[PACKET]
stream_index=0
pts_time=0.140000
duration_time=0.001000
[/PACKET]

[jyavenard]

Original file:
​https://raw.githubusercontent.com/jyavenard/htmltests/master/mediatest/1496641/MozillaTest.webm
remuxed webm
​https://jyavenard.github.io/htmltests/mediatest/1496641/test.webm

transmuxed mp4:
​https://jyavenard.github.io/htmltests/mediatest/1496641/test.mp4

[Gyan]

WebM files, especially when generated via WebRTC, often show the reciprocal of the timebase as their framerate. The timebase for Matroska/WebM files is 1/1000.

The MP4 muxer defaults to constant frame rate, so with your first command, ffmpeg will duplicate frames to achieve a 1000 fps output. In this output, each frame will obviously have duration 1/1000 s.

Use

ffmpeg -i MozilleTest -vsync vfr -c:v libx264 test.mp4

to preserve original timestamps with no frame duplication. The resulting file will have an avg rate of 30.04.

[jyavenard]

would there be a way to detect if the frame is a duplicated one and drop it ?

Otherwise, the default behaviour is pretty silly. The average framerate for that video over its entire duration is 31.96 otherwise.

[Gyan]

To be clear, I meant that ffmpeg will duplicate frames given the frame rate set for the output AVStream, if the muxing mode is CFR. not that the original WebM has duplicated frames. The vsync option lets you control that logic to some extent.

CFR is default to MP4 but not all muxers, and is chosen because of a belief, not fully substantiated to me, that the MP4 muxer does not produce valid VFR files. It's too combersome, right now, to comb through all of ISO 14496-12/14 and identify what the requirements for VFR files are and how we fail to meet them.

However, the primary issue here is the WebM demuxer and/or the conformity of the source video, not vsync or the MP4 muxer.

For what it's worth, I see that neither your MP4 output nor mine shows video in Firefox 62.0.3/Win7 for me.

I can get around that by setting an output rate.

ffmpeg -i MozilleTest -vsync vfr -r 40 -c:v libx264 test.mp4

This will drop frames whose duration < 1/40 s but not duplicate any frames.

[Robert Dowdle]

Replying to Gyan:

For what it's worth, I see that neither your MP4 output nor mine shows video in Firefox 62.0.3/Win7 for me.

If I try to stream the MP4 output file from safari on an Iphone (Iphone 8) device it won't stream. I just get an icon indicating that it can't even play it.

Sorry if this is an aside, but I'm wondering if it's related to the frame issue as well. And it seems you could probably find an answer for me. Appreciate it.

[Carl Eugen Hoyos]

Replying to Gyan:

Use

ffmpeg -i MozilleTest -vsync vfr -c:v libx264 test.mp4

to preserve original timestamps with no frame duplication.

This can produce an invalid mp4 file: 91360ce61d00bb20429fb41857abebf50a5c6b1d

[Gyan]

Replying to cehoyos:

Replying to Gyan:

Use

ffmpeg -i MozilleTest -vsync vfr -c:v libx264 test.mp4

to preserve original timestamps with no frame duplication.

This can produce an invalid mp4 file: 91360ce61d00bb20429fb41857abebf50a5c6b1d

The commit msg says that the muxer supports VFR but sync issues are seen. In fact, that commit message doesn't indicate any standards-based deficiency of the muxer, so output may lead to undesired playback behaviour, but not an 'invalid' file.

Replying to tdowdle:

If I try to stream the MP4 output file from safari on an Iphone (Iphone 8) device it won't stream. I just get an icon indicating that it can't even play it.

Add -movflags +faststart and check.

[Robert Dowdle]

Replying to Gyan:

Add -movflags +faststart and check.

Thank you for your help, today I just got around to testing the commands you gave.

ffmpeg -i MozillaTest -vsync vfr -r 40 -c:v libx264 test.mp4

Seemed to do the trick for ios mobile as well.

Thank you