Encoding image only movie failure

[Jörgen Isaksson]

I'm trying to overlay a QuickTime movie that consists of PNG images on top of another movie. I have noticed that FFMPEG doesn't seem to handle such an image-only movie correctly.

To illustrate the problem I have made a simple example movie that has one image sample for each second with a timecode written to it. When I try to convert this movie and change the framerate FFMPEG seems misinterpret samples randomly.

Download the sample movie from ​http://dl.dropbox.com/u/659266/timecodes.mov and try with the following simple command line:

ffmpeg -y -i timecodes.mov -r 25 timecodes_copy.mov

You will see that FFMPEG will shorten the first sample to 1 frame instead of 25? The sample that should appear on frame 26 will instead appear on frame 2?

Looking forward to some help with this problem!

Best regards / Jorgen

ffmpeg version git-N-29387-gff558d7, Copyright (c) 2000-2011 the FFmpeg developers

built on Apr 26 2011 13:27:41 with clang 2.0 (tags/Apple/clang-139)
configuration: --arch=x86_64 --cc=clang --disable-debug --disable-shared --enable-static --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-libvpx --enable-libtheora --enable-libvorbis --disable-ffplay --disable-ffserver --enable-gpl --enable-swscale --enable-avfilter --enable-filter=movie
libavutil 51. 0. 0 / 51. 0. 0
libavcodec 53. 1. 0 / 53. 1. 0
libavformat 53. 0. 3 / 53. 0. 3
libavdevice 53. 0. 0 / 53. 0. 0
libavfilter 2. 0. 0 / 2. 0. 0
libswscale 0. 13. 0 / 0. 13. 0

[Carl Eugen Hoyos]

Please add complete, uncut output of your call to ffmpeg.

[Jörgen Isaksson]

ffmpeg -y -i /Users/jogga/Desktop/timecodes.mov -r 25 /Users/jogga/Desktop/timecodes_copy.mov
ffmpeg version git-N-29387-gff558d7, Copyright (c) 2000-2011 the FFmpeg developers

built on Apr 26 2011 13:27:41 with clang 2.0 (tags/Apple/clang-139)
configuration: --arch=x86_64 --cc=clang --disable-debug --disable-shared --enable-static --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-libvpx --enable-libtheora --enable-libvorbis --disable-ffplay --disable-ffserver --enable-gpl --enable-swscale --enable-avfilter --enable-filter=movie
libavutil 51. 0. 0 / 51. 0. 0
libavcodec 53. 1. 0 / 53. 1. 0
libavformat 53. 0. 3 / 53. 0. 3
libavdevice 53. 0. 0 / 53. 0. 0
libavfilter 2. 0. 0 / 2. 0. 0
libswscale 0. 13. 0 / 0. 13. 0

Seems stream 0 codec frame rate differs from container frame rate: 600.00 (600/1) -> 1.00 (1/1)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/jogga/Desktop/timecodes.mov':

Metadata:

major_brand : qt
minor_version : 537199360
compatible_brands: qt
creation_time : 2011-04-13 15:15:30

Duration: 00:01:40.00, start: 0.000000, bitrate: 34 kb/s

Stream #0.0(eng): Video: png, bgra, 320x240, 34 kb/s, 1 fps, 1 tbr, 600 tbn, 600 tbc
Metadata:

creation_time : 2011-04-13 15:15:30

Incompatible pixel format 'bgra' for codec 'mpeg4', auto-selecting format 'yuv420p'
[buffer @ 0x101300ea0] w:320 h:240 pixfmt:bgra
[ffsink @ 0x101301100] auto-inserting filter 'auto-inserted scaler 0' between the filter 'src' and the filter 'out'
[scale @ 0x101301330] w:320 h:240 fmt:bgra -> w:320 h:240 fmt:yuv420p flags:0xa0000004
Output #0, mov, to '/Users/jogga/Desktop/timecodes_copy.mov':

Metadata:

major_brand : qt
minor_version : 537199360
compatible_brands: qt
creation_time : 2011-04-13 15:15:30
encoder : Lavf53.0.3
Stream #0.0(eng): Video: mpeg4, yuv420p, 320x240, q=2-31, 200 kb/s, 25 tbn, 25 tbc
Metadata:

creation_time : 2011-04-13 15:15:30

Stream mapping:

Stream #0.0 -> #0.0

Press [q] to stop encoding
frame= 50 fps= 0 q=2.0 size= 46kB time=2.00 bitrate= 190.3kbits/s dup=47frame= 150 fps= 94 q=2.0 size= 126kB time=6.00 bitrate= 172.2kbits/s dup=14frame= 275 fps=127 q=2.0 size= 226kB time=11.00 bitrate= 168.2kbits/s dup=2frame= 350 fps=127 q=2.0 size= 294kB time=14.00 bitrate= 171.8kbits/s dup=3frame= 425 fps=122 q=2.0 size= 352kB time=17.00 bitrate= 169.5kbits/s dup=4frame= 500 fps=120 q=2.0 size= 411kB time=20.00 bitrate= 168.4kbits/s dup=4frame= 600 fps=121 q=2.0 size= 490kB time=24.00 bitrate= 167.4kbits/s dup=5frame= 700 fps=127 q=2.0 size= 578kB time=28.00 bitrate= 169.1kbits/s dup=6frame= 800 fps=133 q=2.0 size= 658kB time=32.00 bitrate= 168.4kbits/s dup=7frame= 850 fps=127 q=2.0 size= 698kB time=34.00 bitrate= 168.1kbits/s dup=8frame= 900 fps=123 q=2.0 size= 738kB time=36.00 bitrate= 167.9kbits/s dup=8frame= 1050 fps=132 q=2.0 size= 864kB time=42.00 bitrate= 168.6kbits/s dup=1frame= 1100 fps=130 q=2.0 size= 904kB time=44.00 bitrate= 168.3kbits/s dup=1frame= 1125 fps=124 q=2.0 size= 924kB time=45.00 bitrate= 168.2kbits/s dup=1frame= 1175 fps=117 q=2.0 size= 963kB time=47.00 bitrate= 167.8kbits/s dup=1frame= 1225 fps=115 q=1.6 size= 1011kB time=49.00 bitrate= 169.0kbits/s dup=1frame= 1275 fps=114 q=2.0 size= 1050kB time=51.00 bitrate= 168.7kbits/s dup=1frame= 1375 fps=116 q=2.0 size= 1130kB time=55.00 bitrate= 168.4kbits/s dup=1frame= 1450 fps=117 q=2.0 size= 1190kB time=58.00 bitrate= 168.1kbits/s dup=1frame= 1550 fps=118 q=2.0 size= 1277kB time=62.00 bitrate= 168.7kbits/s dup=1frame= 1575 fps=112 q=2.0 size= 1297kB time=63.00 bitrate= 168.6kbits/s dup=1frame= 1625 fps=109 q=2.0 size= 1336kB time=65.00 bitrate= 168.4kbits/s dup=1frame= 1675 fps=108 q=2.0 size= 1375kB time=67.00 bitrate= 168.1kbits/s dup=1frame= 1700 fps=104 q=2.0 size= 1395kB time=68.00 bitrate= 168.0kbits/s dup=1frame= 1725 fps= 98 q=2.0 size= 1414kB time=69.00 bitrate= 167.9kbits/s dup=1frame= 1825 fps=101 q=1.6 size= 1501kB time=73.00 bitrate= 168.4kbits/s dup=1frame= 1900 fps=102 q=2.0 size= 1558kB time=76.00 bitrate= 167.9kbits/s dup=1frame= 1925 fps= 99 q=2.0 size= 1577kB time=77.00 bitrate= 167.8kbits/s dup=1frame= 2000 fps= 99 q=2.0 size= 1635kB time=80.00 bitrate= 167.5kbits/s dup=1frame= 2050 fps= 99 q=2.0 size= 1674kB time=82.00 bitrate= 167.2kbits/s dup=1frame= 2100 fps= 99 q=2.0 size= 1713kB time=84.00 bitrate= 167.1kbits/s dup=2frame= 2175 fps= 99 q=2.0 size= 1780kB time=87.00 bitrate= 167.6kbits/s dup=2frame= 2250 fps=100 q=2.0 size= 1839kB time=90.00 bitrate= 167.4kbits/s dup=2frame= 2325 fps=101 q=2.0 size= 1897kB time=93.00 bitrate= 167.1kbits/s dup=2frame= 2400 fps=102 q=2.0 size= 1956kB time=96.00 bitrate= 166.9kbits/s dup=2frame= 2450 fps=101 q=2.0 size= 2003kB time=98.00 bitrate= 167.5kbits/s dup=2frame= 2475 fps=100 q=2.0 Lsize= 2044kB time=99.00 bitrate= 169.1kbits/s dup=2375 drop=0
video:2023kB audio:0kB global headers:0kB muxing overhead 1.031384%

[Carl Eugen Hoyos]

Possibly a muxer problem.

[Jörgen Isaksson]

cehoyos,

I use this mainly to hard-code subtitles into mp4 files. I render the subtitles into a QuickTime movie as PNG images. I then use the movie overlay filter to superimpose the subtitles movie on top of the original movie. This is a very good solution to get hard-coded subtitles using FFMPEG. Unfortunately the above mention bug has made it impossible to use :-(

I'm really looking forward to a solution to this problem. Do you need some more sample movies?

Just let me know if there's anything I can do to help.

BTW, Thx for your quick reply and handling of this matter!

best regards / Jorgen

[Jörgen Isaksson]

I had an older version of FFMPEG pulled from my backup where I remember that this was working. Maybe you could compare the code from the subversion repository to easier find the problem.

FFmpeg version SVN-r25526, Copyright (c) 2000-2010 the FFmpeg developers

built on Nov 16 2010 13:12:40 with clang 1.6 (tags/Apple/clang-70)
configuration: --arch=x86_64 --cc=clang --disable-debug --disable-shared --enable-static --enable-libmp3lame --enable-libx264 --enable-libxvid --disable-ffplay --disable-ffserver --enable-gpl --enable-avfilter --enable-filter=movie
libavutil 50.32. 3 / 50.32. 3
libavcore 0. 9. 1 / 0. 9. 1
libavcodec 52.92. 0 / 52.92. 0
libavformat 52.83. 0 / 52.83. 0
libavdevice 52. 2. 2 / 52. 2. 2
libavfilter 1.53. 0 / 1.53. 0
libswscale 0.12. 0 / 0.12. 0

Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

[Carl Eugen Hoyos]

(Complete, uncut output missing.)

If you believe this is a regression, please find the version introducing it with git bisect.

[Jörgen Isaksson]

But the FFMPEG version I have working is from the SVN repository back somewhere in november 2010. Will I still be able to find that in the GIT repository?

[Carl Eugen Hoyos]

git log --grep="revision 25526" -> 4ad08021e8e81057d89c75fd63b97cd1f0757a23

[Jörgen Isaksson]

Yes!!!

I found the revision intruducing the bug using git bisect as you suggested.

91360ce61d00bb20429fb41857abebf50a5c6b1d is the first bad commit
commit 91360ce61d00bb20429fb41857abebf50a5c6b1d
Author: Baptiste Coudurier <​baptiste.coudurier@gmail.com>
Date: Tue Nov 23 00:41:28 2010 +0000

Unset variable fps for mov and mp4 muxer, they support it
but timestamps must start at 0 currently, and this causes sync
problem.

Originally committed as revision 25805 to ​svn://svn.ffmpeg.org/ffmpeg/trunk

:040000 040000 6c6427cbb95b35f35b8d219ce218a90a8b2582e7 ac7d1a5c490707cfa20fd1f461e0e04c1af8916d M libavformat

It seems like something changed in the mov and mp4 muxer.

How do we continue to solve this bug?

[Jörgen Isaksson]

More info from git show

commit 91360ce61d00bb20429fb41857abebf50a5c6b1d
Author: Baptiste Coudurier <​baptiste.coudurier@gmail.com>
Date: Tue Nov 23 00:41:28 2010 +0000

Unset variable fps for mov and mp4 muxer, they support it
but timestamps must start at 0 currently, and this causes sync
problem.

Originally committed as revision 25805 to ​svn://svn.ffmpeg.org/ffmpeg/trunk

diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 77f122b..b1ad714 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -2215,7 +2215,7 @@ AVOutputFormat mov_muxer = {

mov_write_header,
ff_mov_write_packet,
mov_write_trailer,

• .flags = AVFMT_GLOBALHEADER | AVFMT_VARIABLE_FPS,

+ .flags = AVFMT_GLOBALHEADER,

.codec_tag = (const AVCodecTag* const []){codec_movvideo_tags, codec_movaud

};
#endif

[Jörgen Isaksson]

Now I have compiled a version that can encode the movie correctly by enabling the AVFMT_VARIABLE_FPS flags for the mov and mp4 muxers.

I still can't get the movie overlay to work as expected. When I'm running a command like:

ffmpeg -y -i /Users/jogga/Movies/skiing.m4v -acodec copy -s 640x360 -vcodec libx264 -vpre slow -vpre ipod640 -crf 22 -threads 0 -vf "movie=/Users/jogga/Movies/skiing_subtitles.mov:f=mov, pad=640:360:0:286:0x00000000[subtitles];[in][subtitles] overlay=0:0:1[out]" /Users/jogga/Desktop/Slaskmappen/skiing_subtitled.m4v

it still skips the empty samples at the beginning of the skiing_subtitles.mov file and everything gets out of sync.

This is the output from the above command:

ffmpeg version git-N-29957-g85eedcf, Copyright (c) 2000-2011 the FFmpeg developers

built on May 16 2011 15:37:05 with clang 2.0 (tags/Apple/clang-139)
configuration: --arch=x86_64 --cc=clang --disable-debug --disable-shared --enable-static --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-libvpx --enable-libtheora --enable-libvorbis --disable-ffplay --disable-ffserver --enable-gpl --enable-swscale --enable-avfilter --enable-filter=movie
libavutil 51. 2. 1 / 51. 2. 1
libavcodec 53. 5. 0 / 53. 5. 0
libavformat 53. 1. 0 / 53. 1. 0
libavdevice 53. 0. 0 / 53. 0. 0
libavfilter 2. 5. 0 / 2. 5. 0
libswscale 0. 14. 0 / 0. 14. 0
libpostproc 51. 2. 0 / 51. 2. 0

Seems stream 1 codec frame rate differs from container frame rate: 5000.00 (5000/1) -> 25.00 (25/1)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/jogga/Movies/skiing.m4v':

Metadata:

major_brand : M4VP
minor_version : 1
compatible_brands: M4VPM4A mp42isom
creation_time : 2011-05-16 14:02:59
encoder : CoreMediaAuthoring 706, CoreMedia 484.20, i386
title : skiing.mov

Duration: 00:00:10.03, start: 0.000000, bitrate: 937 kb/s

Stream #0.0(und): Audio: aac, 44100 Hz, stereo, s16, 128 kb/s
Metadata:

creation_time : 2011-05-16 14:02:59

Stream #0.1(und): Video: h264 (Constrained Baseline), yuv420p, 480x270 [PAR 1:1 DAR 16:9], 798 kb/s, 25.02 fps, 25 tbr, 2500 tbn, 5k tbc
Metadata:

creation_time : 2011-05-16 14:02:59

[buffer @ 0x101302000] w:480 h:270 pixfmt:yuv420p tb:1/1000000 sar:1/1
[movie @ 0x101300f80] seek_point:0 format_name:mov file_name:/Users/jogga/Movies/skiing_subtitles.mov stream_index:0
[overlay @ 0x101306780] auto-inserting filter 'auto-inserted scaler 0' between the filter 'Parsed filter 1 pad' and the filter 'Parsed filter 2 overlay'
[scale @ 0x101304b80] w:480 h:270 fmt:yuv420p -> w:640 h:360 fmt:yuv420p flags:0xa0000004
[pad @ 0x101306560] w:640 h:74 -> w:640 h:360 x:0 y:286 color:0x00000000[rgba]
[scale @ 0x101306da0] w:640 h:360 fmt:bgra -> w:640 h:360 fmt:yuva420p flags:0xa0000004
[overlay @ 0x101306780] main w:640 h:360 fmt:yuv420p overlay x:0 y:0 w:640 h:360 fmt:yuva420p
[overlay @ 0x101306780] main_tb:1/1000000 overlay_tb:1/600 -> tb:1/3000000 exact:1
[libx264 @ 0x101801800] using SAR=1/1
[libx264 @ 0x101801800] using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2
[libx264 @ 0x101801800] profile Constrained Baseline, level 3.0
[libx264 @ 0x101801800] 264 - core 115 r1937 aa21558 - H.264/MPEG-4 AVC codec - Copyleft 2003-2011 - ​http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=1:0:0 analyse=0x1:0x111 me=umh subme=8 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=1 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=50 rc=crf mbtree=1 crf=22.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 vbv_maxrate=10000 vbv_bufsize=10000 crf_max=0.0 nal_hrd=none ip_ratio=1.41 aq=1:1.00
Output #0, ipod, to '/Users/jogga/Desktop/Slaskmappen/skiing_subtitled.m4v':

Metadata:

major_brand : M4VP
minor_version : 1
compatible_brands: M4VPM4A mp42isom
creation_time : 2011-05-16 14:02:59
title : skiing.mov
encoder : Lavf53.1.0
Stream #0.0(und): Video: libx264, yuv420p, 640x360 [PAR 1:1 DAR 16:9], q=0-69, 200 kb/s, 25 tbn, 25 tbc
Metadata:

creation_time : 2011-05-16 14:02:59

Stream #0.1(und): Audio: aac, 44100 Hz, stereo, 128 kb/s
Metadata:

creation_time : 2011-05-16 14:02:59

Stream mapping:

Stream #0.1 -> #0.0
Stream #0.0 -> #0.1

Press [q] to stop, ? for help
frame= 250 fps= 66 q=-1.0 Lsize= 1632kB time=10.00 bitrate=1336.7kbits/s
video:1467kB audio:158kB global headers:0kB muxing overhead 0.403333%
frame I:10 Avg QP:18.48 size: 14530
[libx264 @ 0x101801800] frame P:240 Avg QP:23.91 size: 5652
[libx264 @ 0x101801800] mb I I16..4: 46.0% 0.0% 54.0%
[libx264 @ 0x101801800] mb P I16..4: 7.2% 0.0% 5.2% P16..4: 42.5% 18.6% 5.0% 0.0% 0.0% skip:21.5%
[libx264 @ 0x101801800] coded y,uvDC,uvAC intra: 39.4% 25.0% 2.6% inter: 30.5% 17.0% 0.0%
[libx264 @ 0x101801800] i16 v,h,dc,p: 49% 24% 12% 14%
[libx264 @ 0x101801800] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 17% 11% 9% 11% 10% 12% 7% 10%
[libx264 @ 0x101801800] i8c dc,h,v,p: 75% 12% 10% 3%
[libx264 @ 0x101801800] kb/s:1201.48

What could be wrong now?

[Michael Niedermayer]

The ​http://dl.dropbox.com/u/659266/timecodes.mov file is 404

[Jörgen Isaksson]

Thanks for adding the sample movie back again cehoyos. I accidentally removed it from my dropbox.

michael, just let me know if you need any further samples or anything to solve this bug.

[Michael Niedermayer]

The frame rate changing should be working now

[Jörgen Isaksson]

Hi michael,

It now works when changing the frame rate for a single file like in the example at the top above. But when I try to use the movie overlay filter it still doesn't work. The empty samples at the beginning aren't handled correctly.

I will file a new bug for the movie overlay filter. Hopefully it's easy to fix now that you found out what was wrong for the single file case.

[Jörgen Isaksson]

I have filed a new ticket now, Ticket #467

https://ffmpeg.org/trac/ffmpeg/ticket/467