Concatenating images using the framerate option results in unexpected durations

[sycxyc]

Summary of the bug:

I want to output the video with time_base=1/100, the option framerate 100 directive can do this, but it will result in incorrect durations.

How to reproduce:

$ cat test.txt 
ffconcat version 1.0
file https://github.com/fluidicon.png
option framerate 100
duration 0.030
file https://github.com/fluidicon.png
duration 1
file https://github.com/fluidicon.png



$ ffmpeg -protocol_whitelist file,http,https,tcp,tls,crypto -safe 0 -f concat -i test.txt -fps_mode vfr test.mp4 -report
ffmpeg version 2024-03-11-git-3d1860ec8d-essentials_build-www.gyan.dev Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 13.2.0 (Rev5, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --pkg-config=pkgconf --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
  libavutil      59.  1.100 / 59.  1.100
  libavcodec     61.  1.100 / 61.  1.100
  libavformat    61.  0.100 / 61.  0.100
  libavdevice    61.  0.100 / 61.  0.100
  libavfilter    10.  0.100 / 10.  0.100
  libswscale      8.  0.100 /  8.  0.100
  libswresample   5.  0.100 /  5.  0.100
  libpostproc    58.  0.100 / 58.  0.100
Splitting the commandline.
Reading option '-protocol_whitelist' ... matched as AVOption 'protocol_whitelist' with argument 'file,http,https,tcp,tls,crypto'.
Reading option '-safe' ... matched as AVOption 'safe' with argument '0'.
Reading option '-f' ... matched as option 'f' (force container format (auto-detected otherwise)) with argument 'concat'.
Reading option '-i' ... matched as output url with argument 'test.txt'.
Reading option '-fps_mode' ... matched as option 'fps_mode' (set framerate mode for matching video streams; overrides vsync) with argument 'vfr'.
Reading option 'test.mp4' ... matched as output url.
Reading option '-report' ... matched as option 'report' (generate a report) with argument '1'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option report (generate a report) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url test.txt.
Applying option f (force container format (auto-detected otherwise)) with argument concat.
Successfully parsed a group of options.
Opening an input file: test.txt.
[concat @ 00000137e8b71c40] Opening 'test.txt' for reading
[AVFormatContext @ 00000137e8b7b6c0] Opening 'https://github.com/fluidicon.png' for reading
[tcp @ 00000137e8b80100] Original list of addresses:
[tcp @ 00000137e8b80100] Address 20.205.243.166 port 443
[tcp @ 00000137e8b80100] Interleaved list of addresses:
[tcp @ 00000137e8b80100] Address 20.205.243.166 port 443
[tcp @ 00000137e8b80100] Starting connection attempt to 20.205.243.166 port 443
[tcp @ 00000137e8b80100] Successfully connected to 20.205.243.166 port 443
[https @ 00000137e8b7cc00] request: GET /fluidicon.png HTTP/1.1
User-Agent: Lavf/61.0.100
Accept: */*
Range: bytes=0-
Connection: close
Host: github.com
Icy-MetaData: 1


[png_pipe @ 00000137e8b7b6c0] Format png_pipe probed with size=2048 and score=99
[png_pipe @ 00000137e8b7b6c0] Before avformat_find_stream_info() pos: 0 bytes read:3019 seeks:0 nb_streams:1
[png_pipe @ 00000137e8b7b6c0] After avformat_find_stream_info() pos: 33270 bytes read:33270 seeks:0 frames:1
[concat @ 00000137e8b71c40] Before avformat_find_stream_info() pos: 187 bytes read:187 seeks:0 nb_streams:1
[concat @ 00000137e8b71c40] file:0 stream:0 pts:0 pts_time:0 dts:0 dts_time:0 -> pts:0 pts_time:0 dts:0 dts_time:0
[concat @ 00000137e8b71c40] All info found
[concat @ 00000137e8b71c40] After avformat_find_stream_info() pos: 187 bytes read:187 seeks:0 frames:1
Input #0, concat, from 'test.txt':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0, 1, 1/100: Video: png, rgba(pc, gbr/unknown/unknown), 512x512, 100 fps, 100 tbr, 100 tbn
Successfully opened the file.
Parsing a group of options: output url test.mp4.
Applying option fps_mode (set framerate mode for matching video streams; overrides vsync) with argument vfr.
Successfully parsed a group of options.
Opening an output file: test.mp4.
[out#0/mp4 @ 00000137e90cd400] No explicit maps, mapping streams automatically...
[vost#0:0/libx264 @ 00000137e918a180] Created video stream from input stream 0:0
detected 2 logical cores
[file @ 00000137e8b84ec0] Setting default whitelist 'file,crypto,data'
Successfully opened the file.
Stream mapping:
  Stream #0:0 -> #0:0 (png (native) -> h264 (libx264))
[vost#0:0/libx264 @ 00000137e918a180] Starting thread...
[vf#0:0 @ 00000137e91436c0] Starting thread...
[vist#0:0/png @ 00000137e91884c0] [dec:png @ 00000137e91d7f80] Starting thread...
[in#0/concat @ 00000137e8b71840] Starting thread...
Press [q] to stop, [?] for help
[AVIOContext @ 00000137e9137500] Statistics: 33270 bytes read, 0 seeks
[AVFormatContext @ 00000137e91e1d80] Opening 'https://github.com/fluidicon.png' for reading
[tcp @ 00000137e91ddc40] Original list of addresses:
[tcp @ 00000137e91ddc40] Address 20.205.243.166 port 443
[tcp @ 00000137e91ddc40] Interleaved list of addresses:
[tcp @ 00000137e91ddc40] Address 20.205.243.166 port 443
[tcp @ 00000137e91ddc40] Starting connection attempt to 20.205.243.166 port 443
[tcp @ 00000137e91ddc40] Successfully connected to 20.205.243.166 port 443
[https @ 00000137e913d1c0] request: GET /fluidicon.png HTTP/1.1
User-Agent: Lavf/61.0.100
Accept: */*
Range: bytes=0-
Connection: close
Host: github.com
Icy-MetaData: 1


[png_pipe @ 00000137e91e1d80] Format png_pipe probed with size=2048 and score=99
[png_pipe @ 00000137e91e1d80] Before avformat_find_stream_info() pos: 0 bytes read:3018 seeks:0 nb_streams:1
[png_pipe @ 00000137e91e1d80] After avformat_find_stream_info() pos: 33270 bytes read:33270 seeks:0 frames:1
[concat @ 00000137e8b71c40] file:1 stream:0 pts:0 pts_time:0 dts:0 dts_time:0 -> pts:1 pts_time:0.04 dts:1 dts_time:0.04
[AVIOContext @ 00000137e90cbe00] Statistics: 33270 bytes read, 0 seeks
[AVFormatContext @ 00000137e91e1d80] Opening 'https://github.com/fluidicon.png' for reading
[tcp @ 00000137e91ddc40] Original list of addresses:
[tcp @ 00000137e91ddc40] Address 20.205.243.166 port 443
[tcp @ 00000137e91ddc40] Interleaved list of addresses:
[tcp @ 00000137e91ddc40] Address 20.205.243.166 port 443
[tcp @ 00000137e91ddc40] Starting connection attempt to 20.205.243.166 port 443
[tcp @ 00000137e91ddc40] Successfully connected to 20.205.243.166 port 443
[https @ 00000137e913d1c0] request: GET /fluidicon.png HTTP/1.1
User-Agent: Lavf/61.0.100
Accept: */*
Range: bytes=0-
Connection: close
Host: github.com
Icy-MetaData: 1


[png_pipe @ 00000137e91e1d80] Format png_pipe probed with size=2048 and score=99
[png_pipe @ 00000137e91e1d80] Before avformat_find_stream_info() pos: 0 bytes read:3025 seeks:0 nb_streams:1
[png_pipe @ 00000137e91e1d80] After avformat_find_stream_info() pos: 33270 bytes read:33270 seeks:0 frames:1
[concat @ 00000137e8b71c40] file:2 stream:0 pts:0 pts_time:0 dts:0 dts_time:0 -> pts:26 pts_time:1.04 dts:26 dts_time:1.04
[in#0/concat @ 00000137e8b71840] EOF while reading input
[in#0/concat @ 00000137e8b71840] Terminating thread with return code 0 (success)
[vist#0:0/png @ 00000137e91884c0] [dec:png @ 00000137e91d7f80] Decoder thread received EOF packet
[vist#0:0/png @ 00000137e91884c0] [dec:png @ 00000137e91d7f80] Decoder returned EOF, finishing
[vist#0:0/png @ 00000137e91884c0] [dec:png @ 00000137e91d7f80] Terminating thread with return code 0 (success)
[graph 0 input from stream 0:0 @ 00000137e9246f00] Setting 'video_size' to value '512x512'
[graph 0 input from stream 0:0 @ 00000137e9246f00] Setting 'pix_fmt' to value '26'
[graph 0 input from stream 0:0 @ 00000137e9246f00] Setting 'time_base' to value '1/100'
[graph 0 input from stream 0:0 @ 00000137e9246f00] Setting 'pixel_aspect' to value '0/1'
[graph 0 input from stream 0:0 @ 00000137e9246f00] Setting 'colorspace' to value '0'
[graph 0 input from stream 0:0 @ 00000137e9246f00] Setting 'range' to value '2'
[graph 0 input from stream 0:0 @ 00000137e9246f00] Setting 'frame_rate' to value '100/1'
[graph 0 input from stream 0:0 @ 00000137e9246f00] w:512 h:512 pixfmt:rgba tb:1/100 fr:100/1 sar:0/1 csp:gbr range:pc
[format @ 00000137e8be3180] Setting 'pix_fmts' to value 'yuv420p|yuvj420p|yuv422p|yuvj422p|yuv444p|yuvj444p|nv12|nv16|nv21|yuv420p10le|yuv422p10le|yuv444p10le|nv20le|gray|gray10le'
[auto_scale_0 @ 00000137e8be3780] w:iw h:ih flags:'' interl:0
[format @ 00000137e8be3180] auto-inserting filter 'auto_scale_0' between the filter 'Parsed_null_0' and the filter 'format'
[AVFilterGraph @ 00000137e8b860c0] query_formats: 4 queried, 6 merged, 3 already done, 0 delayed
[auto_scale_0 @ 00000137e8be3780] picking yuv444p out of 14 ref:rgba alpha:1
[auto_scale_0 @ 00000137e8be3780] w:512 h:512 fmt:rgba csp:gbr range:pc sar:0/1 -> w:512 h:512 fmt:yuv444p csp:unknown range:unknown sar:0/1 flags:0x00000004
[graph 0 input from stream 0:0 @ 00000137e9246f00] video frame properties congruent with link at pts_time: 0
[libx264 @ 00000137e9143340] using mv_range_thread = 88
[libx264 @ 00000137e9143340] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
[libx264 @ 00000137e9143340] profile High 4:4:4 Predictive, level 3.1, 4:4:4, 8-bit
[libx264 @ 00000137e9143340] 264 - core 164 r3182 be4f020 - H.264/MPEG-4 AVC codec - Copyleft 2003-2024 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'test.mp4':
  Metadata:
    encoder         : Lavf61.0.100
  Stream #0:0, 0, 1/12800: Video: h264 (avc1 / 0x31637661), yuv444p(progressive), 512x512, q=2-31, 100 fps, 12800 tbn
      Metadata:
        encoder         : Lavc61.1.100 libx264
      Side data:
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
[out#0/mp4 @ 00000137e90cd400] Starting thread...
[out_0_0 @ 00000137e9142200] EOF on sink link out_0_0:default.
[vf#0:0 @ 00000137e91436c0] Filtergraph returned EOF, finishing
[vf#0:0 @ 00000137e91436c0] All consumers returned EOF
[vf#0:0 @ 00000137e91436c0] Terminating thread with return code 0 (success)
[vost#0:0/libx264 @ 00000137e918a180] Encoder thread received EOF
[libx264 @ 00000137e9143340] frame=   0 QP=19.06 NAL=3 Slice:I Poc:0   I:1024 P:0    SKIP:0    size=7731 bytes
[libx264 @ 00000137e9143340] frame=   1 QP=13.57 NAL=2 Slice:P Poc:4   I:79   P:118  SKIP:827  size=460 bytes
[libx264 @ 00000137e9143340] frame=   2 QP=10.20 NAL=0 Slice:B Poc:2   I:18   P:187  SKIP:819  size=243 bytes
[vost#0:0/libx264 @ 00000137e918a180] Terminating thread with return code 0 (success)
[out#0/mp4 @ 00000137e90cd400] All streams finished
[out#0/mp4 @ 00000137e90cd400] Terminating thread with return code 0 (success)
[AVIOContext @ 00000137e91df980] Statistics: 10036 bytes written, 2 seeks, 4 writeouts
[out#0/mp4 @ 00000137e90cd400] Output file #0 (test.mp4):
[out#0/mp4 @ 00000137e90cd400]   Output stream #0:0 (video): 3 frames encoded; 3 packets muxed (9122 bytes); 
[out#0/mp4 @ 00000137e90cd400]   Total: 3 packets (9122 bytes) muxed
[out#0/mp4 @ 00000137e90cd400] video:9KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 9.975882%
frame=    3 fps=2.4 q=-1.0 Lsize=      10KiB time=00:00:00.01 bitrate=8025.6kbits/s speed=0.00788x    
[libx264 @ 00000137e9143340] frame I:1     Avg QP:19.06  size:  7731
[libx264 @ 00000137e9143340] frame P:1     Avg QP:13.57  size:   460
[libx264 @ 00000137e9143340] frame B:1     Avg QP:10.20  size:   243
[libx264 @ 00000137e9143340] consecutive B-frames: 33.3% 66.7%  0.0%  0.0%
[libx264 @ 00000137e9143340] mb I  I16..4: 35.1% 34.0% 31.0%
[libx264 @ 00000137e9143340] mb P  I16..4:  4.9%  2.8%  0.0%  P16..4:  9.3%  2.1%  0.1%  0.0%  0.0%    skip:80.8%
[libx264 @ 00000137e9143340] mb B  I16..4:  1.8%  0.0%  0.0%  B16..8: 17.7%  0.6%  0.0%  direct: 0.0%  skip:80.0%  L0: 7.5% L1:92.5% BI: 0.0%
[libx264 @ 00000137e9143340] 8x8 transform intra:33.6% inter:85.7%
[libx264 @ 00000137e9143340] coded y,u,v intra: 14.8% 0.1% 1.2% inter: 0.2% 0.2% 0.2%
[libx264 @ 00000137e9143340] i16 v,h,dc,p: 41% 51%  7%  0%
[libx264 @ 00000137e9143340] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 51% 29%  0%  0%  0%  0%  0%  0%
[libx264 @ 00000137e9143340] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 27% 24% 30%  4%  3%  3%  3%  3%  3%
[libx264 @ 00000137e9143340] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 00000137e9143340] kb/s:132.30
[in#0/concat @ 00000137e8b71840] Input file #0 (test.txt):
[in#0/concat @ 00000137e8b71840]   Input stream #0:0 (video): 3 packets read (99810 bytes); 3 frames decoded; 0 decode errors; 
[in#0/concat @ 00000137e8b71840]   Total: 3 packets (99810 bytes) demuxed
[AVIOContext @ 00000137e8b83380] Statistics: 33270 bytes read, 0 seeks
[AVIOContext @ 00000137e8b72500] Statistics: 187 bytes read, 0 seeks



$ ffprobe -i test.mp4 -show_entries frame=pts_time
ffprobe version 2024-03-11-git-3d1860ec8d-essentials_build-www.gyan.dev Copyright (c) 2007-2024 the FFmpeg developers
  built with gcc 13.2.0 (Rev5, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --pkg-config=pkgconf --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
  libavutil      59.  1.100 / 59.  1.100
  libavcodec     61.  1.100 / 61.  1.100
  libavformat    61.  0.100 / 61.  0.100
  libavdevice    61.  0.100 / 61.  0.100
  libavfilter    10.  0.100 / 10.  0.100
  libswscale      8.  0.100 /  8.  0.100
  libswresample   5.  0.100 /  5.  0.100
  libpostproc    58.  0.100 / 58.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf61.0.100
  Duration: 00:00:00.27, start: 0.000000, bitrate: 297 kb/s
  Stream #0:0[0x1](und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv444p(progressive), 512x512, 270 kb/s, 11.11 fps, 8 tbr, 12800 tbn (default)
      Metadata:
        handler_name    : VideoHandler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.1.100 libx264
[FRAME]
pts_time=0.000000
[SIDE_DATA]
[/SIDE_DATA]
[/FRAME]
[FRAME]
pts_time=0.010000
[/FRAME]
[FRAME]
pts_time=0.260000
[/FRAME]

Due to this quirk, it is necessary to multiply the duration by 4 (=100/25).

$ cat test2.txt 
ffconcat version 1.0
file https://github.com/fluidicon.png
option framerate 100
duration 0.120
file https://github.com/fluidicon.png
duration 4
file https://github.com/fluidicon.png



$ ffprobe -protocol_whitelist "file,http,https,tcp,tls,crypto" -safe 0 -f concat -i test2.txt -show_entries frame=pts_time
ffprobe version 2024-03-11-git-3d1860ec8d-essentials_build-www.gyan.dev Copyright (c) 2007-2024 the FFmpeg developers
  built with gcc 13.2.0 (Rev5, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --pkg-config=pkgconf --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
  libavutil      59.  1.100 / 59.  1.100
  libavcodec     61.  1.100 / 61.  1.100
  libavformat    61.  0.100 / 61.  0.100
  libavdevice    61.  0.100 / 61.  0.100
  libavfilter    10.  0.100 / 10.  0.100
  libswscale      8.  0.100 /  8.  0.100
  libswresample   5.  0.100 /  5.  0.100
  libpostproc    58.  0.100 / 58.  0.100
Input #0, concat, from 'test2.txt':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: png, rgba(pc, gbr/unknown/unknown), 512x512, 100 fps, 100 tbr, 100 tbn
[FRAME]
pts_time=0.000000
[/FRAME]
[FRAME]
pts_time=0.030000
[/FRAME]
[FRAME]
pts_time=1.030000
[/FRAME]

[Cigaes]

If you have one frame and the frame rate is 100 frames per second, then the duration is 1/100 of second, this is not a limitation of FFmpeg, it is a limitation of mathematics.

[Gyan]

You have to set the same framerate directive for all inputs.

[sycxyc]

Replying to Gyan:

You have to set the same framerate directive for all inputs.

Thanks, set the same framerate directive for each frame worked for me.

Feel free to close this ticket if you don't think this is a bug.

[Gyan]

Strictly speaking, there is a bug, but it has been useful to exploit it in the past to rescale video speed, so there wasn't much incentive to correct it.

*the bug is that the concat demuxer calculates a delta value for the exported pts scaled to the current input's stream time base, however the exported pts is denominated in terms of the first input's stream time base, so the delta when added to the exported pts results in an incorrect timestamp (when the input streams don't share the same TB).