ffmpeg unable to handle input frames of different sizes when setting input frame rate

[TheSHEEEP]

I am trying to create a video from a range of (ordered) images.
If all images are of the same size, this works well. However, if the input images are of a different size, ffmpeg just ignores each frame after the size change.

I tried following this ( ​http://superuser.com/questions/628827/can-ffmpeg-encode-video-from-frames-of-different-sizes ) to apply a scale filter, but the result is the same.

The only way I managed to get all input frames to the video is to remove the "-r 30" from the start. But then, the frame rate is off and I get a 12 second video instead of the 10 it should be.

Funny thing is that FFmpeg correctly recognizes the target duration ( Duration: 00:00:10.00 ) just to stop encoding after ~4 seconds when the size change happens.

ffmpeg -r 30 -f image2 -i 1469527404_%04d.jpg -vf "scale=1280:600" -vcodec libx264 -crf 23 -g 15 -preset veryfast -pix_fmt yuv420p -y test.mp4
ffmpeg version N-77715-gfc703f5 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
  libavutil      55. 12.100 / 55. 12.100
  libavcodec     57. 21.100 / 57. 21.100
  libavformat    57. 21.101 / 57. 21.101
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6. 23.100 /  6. 23.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
[mjpeg @ 00000197d6c9c960] Changing bps to 8
Input #0, image2, from '1469527404_%04d.jpg':
  Duration: 00:00:10.00, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 1280x600 [SAR 1:1 DAR 32:15], 30 fps, 30 tbr, 30 tbn, 30 tbc
[swscaler @ 00000197d7c8b940] deprecated pixel format used, make sure you did set range correctly
[libx264 @ 00000197d6cac0a0] using SAR=1/1
[libx264 @ 00000197d6cac0a0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 00000197d6cac0a0] profile High, level 3.1
[libx264 @ 00000197d6cac0a0] 264 - core 148 r2638 7599210 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=6 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=15 keyint_min=1 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'test.mp4':
  Metadata:
    encoder         : Lavf57.21.101
    Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 1280x600 [SAR 1:1 DAR 32:15], q=-1--1, 30 fps, 15360 tbn, 30 tbc
    Metadata:
      encoder         : Lavc57.21.100 libx264
    Side data:
      unknown side data type 10 (24 bytes)
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
Press [q] to stop, [?] for help
Input stream #0:0 frame changed from size:1280x600 fmt:yuvj420p to size:1366x768 fmt:gray.57x
Input stream #0:0 frame changed from size:1366x768 fmt:gray to size:1366x768 fmt:yuvj420prop=7 speed=2.61x
[swscaler @ 00000197d7835200] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 00000197d6d49700] deprecated pixel format used, make sure you did set range correctly
frame=  148 fps= 43 q=-1.0 Lsize=     531kB time=00:00:04.86 bitrate= 893.4kbits/s dup=0 drop=152 speed=1.42x
video:528kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.482180%
[libx264 @ 00000197d6cac0a0] frame I:10    Avg QP:17.15  size: 40485
[libx264 @ 00000197d6cac0a0] frame P:42    Avg QP:19.71  size:  2676
[libx264 @ 00000197d6cac0a0] frame B:96    Avg QP:20.92  size:   239
[libx264 @ 00000197d6cac0a0] consecutive B-frames: 10.8%  8.1%  0.0% 81.1%
[libx264 @ 00000197d6cac0a0] mb I  I16..4: 27.6% 52.6% 19.8%
[libx264 @ 00000197d6cac0a0] mb P  I16..4:  3.2%  4.4%  0.1%  P16..4:  8.6%  4.1%  1.7%  0.0%  0.0%    skip:77.9%
[libx264 @ 00000197d6cac0a0] mb B  I16..4:  0.2%  0.1%  0.0%  B16..8:  1.5%  0.3%  0.0%  direct: 0.8%  skip:97.0%  L0:47.0% L1:52.4% BI: 0.6%
[libx264 @ 00000197d6cac0a0] 8x8 transform intra:53.4% inter:71.0%
[libx264 @ 00000197d6cac0a0] coded y,uvDC,uvAC intra: 46.5% 14.7% 1.0% inter: 0.6% 1.1% 0.0%
[libx264 @ 00000197d6cac0a0] i16 v,h,dc,p: 24% 67%  7%  2%
[libx264 @ 00000197d6cac0a0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 38% 30% 24%  2%  1%  1%  1%  1%  1%
[libx264 @ 00000197d6cac0a0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 27% 44%  9%  2%  3%  3%  4%  2%  5%
[libx264 @ 00000197d6cac0a0] i8c dc,h,v,p: 67% 16% 16%  1%
[libx264 @ 00000197d6cac0a0] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 00000197d6cac0a0] kb/s:875.99

[TheSHEEEP]

Okay... it works fine when I replace the "-r 30" with "-framerate 30". Don't even need the scale filter in that case, then.

So... There is a difference between "-r" and "-framerate"?
Now that I know it, it also says so in the documentation...

However, maybe a warning would be in order here, because I don't know any person who would expect there to be a difference between r/framerate in this case.

[Carl Eugen Hoyos]

Replying to TheSHEEEP:

I am trying to create a video from a range of (ordered) images.
If all images are of the same size, this works well.

However, if the input images are of a different size, ffmpeg just ignores each frame after the size change.

I cannot reproduce this, please provide input samples.

The input option -r is simply another option than the option -framerate of the image demuxer.

maybe a warning would be in order here

When should this be shown? There is an input option -r that can be used for all input streams, -framerate only exists for a few demuxers (and is a different option).

[Carl Eugen Hoyos]

Please reopen this ticket if you can provide a set of samples that allows to reproduce the "each frame after the size change gets ignored" case, I cannot reproduce.

[LaurentX]

I've attached a set of images to reproduce a similar symptom.
-r option seems to ignore the black frames while -framerate does take them into account in the output mp4.
All the frames have the same size but some have a fmt of yuvj420p while others are yuvj444p.
complete commandline and output are as follow:

ffmpeg -r 5 -i movie_input\%04d.jpg -y -vcodec libx264 -filter_complex [0:v]scale='if(gt(a,1.5),-2,1620)':'if(gt(a,1.5),1080,-2)',setsar=1:1[sout];[sout]crop=1620:1080 -codec:a copy -r 25 -pix_fmt yuv420p -preset ultrafast movie_temp.mp4

vs

ffmpeg -framerate 5 -i movie_input\%04d.jpg -y -vcodec libx264 -filter_complex [0:v]scale='if(gt(a,1.5),-2,1620)':'if(gt(a,1.5),1080,-2)',setsar=1:1[sout];[sout]crop=1620:1080 -codec:a copy -r 25 -pix_fmt yuv420p -preset ultrafast movie_temp.mp4

verbose:

ffmpeg version N-81609-g7b3bc36 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-libebur128 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
  libavutil      55. 29.100 / 55. 29.100
  libavcodec     57. 55.101 / 57. 55.101
  libavformat    57. 48.103 / 57. 48.103
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 61.100 /  6. 61.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
Input #0, image2, from 'movie_input\movie_input\%04d.jpg':
  Duration: 00:00:04.20, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 2464x1647 [SAR 1:1 DAR 2464:1647], 5 fps, 5 tbr, 5 tbn, 5 tbc
[Parsed_setsar_1 @ 0000000000f7ce20] num:den syntax is deprecated, please use num/den or named options instead
[Parsed_setsar_1 @ 0000000000f7c960] num:den syntax is deprecated, please use num/den or named options instead
[swscaler @ 0000000000e3ef00] deprecated pixel format used, make sure you did set range correctly
[libx264 @ 0000000000e84cc0] using SAR=1/1
[libx264 @ 0000000000e84cc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 0000000000e84cc0] profile Constrained Baseline, level 4.0
[libx264 @ 0000000000e84cc0] 264 - core 148 r2705 3f5ed56 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[mp4 @ 0000000000e83d80] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
Output #0, mp4, to 'movie_temp.mp4':
  Metadata:
    encoder         : Lavf57.48.103
    Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 1620x1080 [SAR 1:1 DAR 3:2], q=-1--1, 25 fps, 12800 tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.55.101 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 (mjpeg) -> scale
  crop -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
Input stream #0:0 frame changed from size:2464x1647 fmt:yuvj420p to size:2464x1647 fmt:yuvj444p
[Parsed_setsar_1 @ 0000000002d05960] num:den syntax is deprecated, please use num/den or named options instead
[swscaler @ 00000000028f2de0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0000000000e3eb00] deprecated pixel format used, make sure you did set range correctly
Input stream #0:0 frame changed from size:2464x1647 fmt:yuvj444p to size:2464x1647 fmt:yuvj420p
[Parsed_setsar_1 @ 0000000002d05f00] num:den syntax is deprecated, please use num/den or named options instead
[swscaler @ 00000000028f2de0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 000000000bab22a0] deprecated pixel format used, make sure you did set range correctly
Input stream #0:0 frame changed from size:2464x1647 fmt:yuvj420p to size:2464x1647 fmt:yuvj444p
[Parsed_setsar_1 @ 0000000002d05a20] num:den syntax is deprecated, please use num/den or named options instead
[swscaler @ 0000000000e75720] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0000000000e31460] deprecated pixel format used, make sure you did set range correctly
Input stream #0:0 frame changed from size:2464x1647 fmt:yuvj444p to size:2464x1647 fmt:yuvj420p
[Parsed_setsar_1 @ 0000000002d05d60] num:den syntax is deprecated, please use num/den or named options instead
[swscaler @ 000000000bab00a0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 00000000027bb500] deprecated pixel format used, make sure you did set range correctly
Input stream #0:0 frame changed from size:2464x1647 fmt:yuvj420p to size:2464x1647 fmt:yuvj444p
[Parsed_setsar_1 @ 0000000002d05620] num:den syntax is deprecated, please use num/den or named options instead
[swscaler @ 000000000bab00a0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0000000000e31460] deprecated pixel format used, make sure you did set range correctly
Input stream #0:0 frame changed from size:2464x1647 fmt:yuvj444p to size:2464x1647 fmt:yuvj420p
[Parsed_setsar_1 @ 0000000002d06240] num:den syntax is deprecated, please use num/den or named options instead
[swscaler @ 000000000bab00a0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0000000002792900] deprecated pixel format used, make sure you did set range correctly
frame=  105 fps=0.0 q=-1.0 Lsize=     792kB time=00:00:04.16 bitrate=1558.6kbits/s dup=84 drop=0 speed= 6.9x
video:790kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.153103%
[libx264 @ 0000000000e84cc0] frame I:1     Avg QP:20.00  size: 78151
[libx264 @ 0000000000e84cc0] frame P:104   Avg QP:14.02  size:  7024
[libx264 @ 0000000000e84cc0] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0000000000e84cc0] mb P  I16..4:  7.2%  0.0%  0.0%  P16..4: 10.1%  0.0%  0.0%  0.0%  0.0%    skip:82.7%
[libx264 @ 0000000000e84cc0] coded y,uvDC,uvAC intra: 11.0% 20.9% 10.8% inter: 4.0% 4.1% 1.6%
[libx264 @ 0000000000e84cc0] i16 v,h,dc,p: 85%  7%  4%  5%
[libx264 @ 0000000000e84cc0] i8c dc,h,v,p: 79%  8% 11%  2%
[libx264 @ 0000000000e84cc0] kb/s:1540.30