Opened 3 years ago
Last modified 6 months ago
#9210 new enhancement
Concat images produces wrong timings when duration is in milliseconds
Reported by: | demet64 | Owned by: | |
---|---|---|---|
Priority: | wish | Component: | avformat |
Version: | git-master | Keywords: | concat |
Cc: | sycxyc | Blocked By: | |
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description (last modified by )
This issue was also mentioned here: https://www.reddit.com/r/ffmpeg/comments/e1o3hv/ffconcat_filter_with_millisecond_precision_how/ and here: https://forum.videohelp.com/threads/397470-Converting-a-sequence-of-images-to-video-with-variable-framerate
Summary of the bug:
How to reproduce:
A concat file like this
file 000000.jpg duration 116ms file 000001.jpg duration 116ms file 000002.jpg duration 116ms file 000003.jpg duration 116ms file 000004.jpg duration 116ms file 000005.jpg duration 116ms file 000006.jpg duration 116ms file 000007.jpg duration 116ms file 000008.jpg duration 116ms file 000009.jpg duration 1002ms file 000010.jpg duration 116ms file 000011.jpg duration 116ms file 000012.jpg duration 116ms file 000013.jpg duration 1129ms file 000014.jpg duration 73ms file 000015.jpg duration 73ms file 000016.jpg duration 116ms file 000017.jpg duration 116ms file 000018.jpg duration 2184ms file 000018.jpg
The output:
ffprobe -f concat -i concat.txt -show_frames | findstr "pkt_pts" pkt_pts=0 pkt_pts_time=0.000000 pkt_pts=3 pkt_pts_time=0.120000 pkt_pts=6 pkt_pts_time=0.240000 pkt_pts=9 pkt_pts_time=0.360000 pkt_pts=12 pkt_pts_time=0.480000 pkt_pts=15 pkt_pts_time=0.600000 pkt_pts=17 pkt_pts_time=0.680000 pkt_pts=20 pkt_pts_time=0.800000 pkt_pts=23 pkt_pts_time=0.920000 pkt_pts=26 pkt_pts_time=1.040000 pkt_pts=51 pkt_pts_time=2.040000 pkt_pts=54 pkt_pts_time=2.160000 pkt_pts=57 pkt_pts_time=2.280000 pkt_pts=60 pkt_pts_time=2.400000 pkt_pts=88 pkt_pts_time=3.520000 pkt_pts=90 pkt_pts_time=3.600000 pkt_pts=92 pkt_pts_time=3.680000 pkt_pts=95 pkt_pts_time=3.800000 pkt_pts=98 pkt_pts_time=3.920000 pkt_pts=152 pkt_pts_time=6.080000
The output looks correct at first, but timestamps are rounded, and for some reasons the duration here
pkt_pts_time=0.600000 pkt_pts=17 pkt_pts_time=0.680000
is 80ms instead of 120ms (and expected 116ms)
It's not the problem of ffprobe, when converting to webm, for example, or gif
ffmpeg -i concat.txt -c:v libvpx-vp9 -lossless 1 out.webm
timings would be like showed above.
The other case:
file 000000.jpg duration 50ms file 000001.jpg duration 60ms file 000002.jpg duration 50ms file 000003.jpg duration 60ms file 000004.jpg duration 50ms file 000005.jpg duration 60ms file 000006.jpg duration 50ms file 000007.jpg duration 60ms file 000008.jpg duration 60ms file 000009.jpg duration 50ms file 000010.jpg duration 60ms file 000011.jpg duration 50ms file 000012.jpg duration 60ms file 000013.jpg duration 50ms file 000014.jpg duration 60ms file 000015.jpg duration 50ms file 000016.jpg duration 60ms file 000017.jpg duration 60ms file 000018.jpg duration 50ms file 000019.jpg duration 60ms file 000020.jpg duration 50ms file 000021.jpg duration 60ms file 000022.jpg duration 50ms file 000023.jpg duration 60ms file 000023.jpg
Output:
pkt_pts=0 pkt_pts_time=0.000000 pkt_pts=1 pkt_pts_time=0.040000 pkt_pts=3 pkt_pts_time=0.120000 pkt_pts=4 pkt_pts_time=0.160000 pkt_pts=6 pkt_pts_time=0.240000 pkt_pts=7 pkt_pts_time=0.280000 pkt_pts=8 pkt_pts_time=0.320000 pkt_pts=10 pkt_pts_time=0.400000 pkt_pts=11 pkt_pts_time=0.440000 pkt_pts=13 pkt_pts_time=0.520000 pkt_pts=14 pkt_pts_time=0.560000 pkt_pts=15 pkt_pts_time=0.600000 pkt_pts=17 pkt_pts_time=0.680000 pkt_pts=18 pkt_pts_time=0.720000 pkt_pts=19 pkt_pts_time=0.760000 pkt_pts=21 pkt_pts_time=0.840000 pkt_pts=22 pkt_pts_time=0.880000 pkt_pts=24 pkt_pts_time=0.960000 pkt_pts=25 pkt_pts_time=1.000000 pkt_pts=26 pkt_pts_time=1.040000 pkt_pts=28 pkt_pts_time=1.120000 pkt_pts=29 pkt_pts_time=1.160000 pkt_pts=31 pkt_pts_time=1.240000 pkt_pts=32 pkt_pts_time=1.280000 pkt_pts=33 pkt_pts_time=1.320000
Attachments (2)
Change History (12)
comment:1 by , 3 years ago
Description: | modified (diff) |
---|
comment:2 by , 3 years ago
comment:3 by , 3 years ago
It seems as if this is due to rounding to a timebase of 1/25 somewhere.
This is what person from reddit thread I mentioned says too. The problem is that it happens on input stream and it seems there's no way you can specify timebase of the input stream (like you can set -r
option on input for example)
comment:4 by , 3 years ago
Please (test current FFmpeg git head and) provide the command line you tested together with the complete, uncut console output and the necessary input files to make this a valid ticket.
comment:5 by , 3 years ago
Keywords: | concat added |
---|
by , 3 years ago
Attachment: | ffmpeg-20210501-201727.log added |
---|
by , 3 years ago
Attachment: | concat.txt added |
---|
comment:6 by , 3 years ago
I don't know if input images are necessary. I tried the same concat file with different sets of pictures and it didn't have any impact on timecodes.
follow-up: 8 comment:7 by , 3 years ago
The unwanted 25 fps does seem to be coming from concat
. I tested that along with image2
and found that image2
honors the frame rate passed as a cli option.
Here's a script that creates some sample images, does a concat run and an image2 run, and collects reports for each.
#!/bin/zsh OUT=$(mktemp -d) CATFILE=${OUT}/catfile.txt pushd ${OUT} echo "generating some sample images with Imagemagik's 'convert' tool" for i in {1..60} do IMG=image-$(printf "%03d" ${i}).jpg convert -size 600x300 -gravity center -background black \ -fill purple label:${i} -units PixelsPerInch -density 254 ${IMG} echo "file '${IMG}'" >> ${CATFILE} echo "duration 0.016" >> ${CATFILE} done echo "Wrote sample ffmpeg concat input file: ${CATFILE}" CONCAT_CMD='ffmpeg -f concat -i ${CATFILE} -report concat.mkv > /dev/null' export FFREPORT=file=ffprobe-concat.log echo "Probing concat with:\n${CONCAT_CMD}" eval ${CONCAT_CMD} echo "Wrote video ${OUT}/concat.mkv" echo "Reading frame rate of concat.mkv..." unset FFREPORT ffprobe -v error -show_format -show_streams concat.mkv | grep 'frame_rate' IMAGE2_CMD="ffmpeg -f image2 -framerate 60 -i image-%03d.jpg -report image2.mkv > /dev/null" echo "\nProbing image2 with:\n${IMAGE2_CMD}" export FFREPORT=file=ffprobe-image2.log eval ${IMAGE2_CMD} echo "Wrote video ${OUT}/image2.mkv" echo "Reading frame rate of concat.mkv..." unset FFREPORT ffprobe -v error -show_format -show_streams image2.mkv | grep 'frame_rate' echo "\nFull reports for each run: " ls -1 ${OUT}/ffprobe*.log open ${OUT} popd
When run on my machine, it looks like this:
(base) andre@boom matlab_image_tsp % ./duration_snooping.sh generating some sample images with Imagemagik's 'convert' tool Wrote sample ffmpeg concat input file: /var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/catfile.txt Probing concat with: ffmpeg -f concat -i ${CATFILE} -report concat.mkv > /dev/null ffmpeg started on 2021-12-06 at 04:07:22 Report written to "ffprobe-concat.log" Log level: 48 ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers built with Apple clang version 13.0.0 (clang-1300.0.29.3) configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/4.4.1_3 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-avresample --enable-videotoolbox libavutil 56. 70.100 / 56. 70.100 libavcodec 58.134.100 / 58.134.100 libavformat 58. 76.100 / 58. 76.100 libavdevice 58. 13.100 / 58. 13.100 libavfilter 7.110.100 / 7.110.100 libavresample 4. 0. 0 / 4. 0. 0 libswscale 5. 9.100 / 5. 9.100 libswresample 3. 9.100 / 3. 9.100 libpostproc 55. 9.100 / 55. 9.100 Input #0, concat, from '/var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/catfile.txt': Duration: 00:00:00.96, start: 0.000000, bitrate: 18 kb/s Stream #0:0: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 600x300 [SAR 254:254 DAR 2:1], 25 fps, 25 tbr, 25 tbn, 25 tbc Stream mapping: Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264)) Press [q] to stop, [?] for help [libx264 @ 0x12b815400] using SAR=1/1 [libx264 @ 0x12b815400] using cpu capabilities: ARMv8 NEON [libx264 @ 0x12b815400] profile High 4:4:4 Predictive, level 2.1, 4:4:4, 8-bit [libx264 @ 0x12b815400] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=9 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, matroska, to 'concat.mkv': Metadata: encoder : Lavf58.76.100 Stream #0:0: Video: h264 (H264 / 0x34363248), yuvj444p(pc, bt470bg/unknown/unknown, progressive), 600x300 [SAR 254:254 DAR 2:1], q=2-31, 25 fps, 1k tbn Metadata: encoder : Lavc58.134.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A frame= 26 fps=0.0 q=-1.0 Lsize= 14kB time=00:00:00.92 bitrate= 121.6kbits/s dup=0 drop=34 speed=13.4x video:13kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 6.463387% [libx264 @ 0x12b815400] frame I:1 Avg QP:19.46 size: 495 [libx264 @ 0x12b815400] frame P:12 Avg QP:17.97 size: 657 [libx264 @ 0x12b815400] frame B:13 Avg QP:22.41 size: 314 [libx264 @ 0x12b815400] consecutive B-frames: 11.5% 61.5% 11.5% 15.4% [libx264 @ 0x12b815400] mb I I16..4: 4.2% 93.6% 2.2% [libx264 @ 0x12b815400] mb P I16..4: 1.6% 2.6% 1.4% P16..4: 8.6% 3.2% 0.5% 0.0% 0.0% skip:82.1% [libx264 @ 0x12b815400] mb B I16..4: 0.8% 1.3% 0.3% B16..8: 9.7% 2.8% 0.1% direct: 0.0% skip:84.9% L0:38.7% L1:60.7% BI: 0.7% [libx264 @ 0x12b815400] 8x8 transform intra:71.4% inter:27.3% [libx264 @ 0x12b815400] coded y,u,v intra: 5.0% 4.1% 4.5% inter: 0.5% 0.3% 0.4% [libx264 @ 0x12b815400] i16 v,h,dc,p: 53% 38% 8% 0% [libx264 @ 0x12b815400] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 31% 9% 58% 1% 0% 0% 0% 0% 0% [libx264 @ 0x12b815400] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 21% 47% 3% 2% 2% 2% 1% 2% [libx264 @ 0x12b815400] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x12b815400] ref P L0: 30.0% 7.6% 51.8% 10.6% [libx264 @ 0x12b815400] ref B L0: 75.2% 24.0% 0.7% [libx264 @ 0x12b815400] ref B L1: 96.8% 3.2% [libx264 @ 0x12b815400] kb/s:95.87 Wrote video /var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/concat.mkv Reading frame rate of concat.mkv... r_frame_rate=25/1 avg_frame_rate=25/1 Probing image2 with: ffmpeg -f image2 -framerate 60 -i image-%03d.jpg -report image2.mkv > /dev/null ffmpeg started on 2021-12-06 at 04:07:22 Report written to "ffprobe-image2.log" Log level: 48 ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers built with Apple clang version 13.0.0 (clang-1300.0.29.3) configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/4.4.1_3 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-avresample --enable-videotoolbox libavutil 56. 70.100 / 56. 70.100 libavcodec 58.134.100 / 58.134.100 libavformat 58. 76.100 / 58. 76.100 libavdevice 58. 13.100 / 58. 13.100 libavfilter 7.110.100 / 7.110.100 libavresample 4. 0. 0 / 4. 0. 0 libswscale 5. 9.100 / 5. 9.100 libswresample 3. 9.100 / 3. 9.100 libpostproc 55. 9.100 / 55. 9.100 Input #0, image2, from 'image-%03d.jpg': Duration: 00:00:01.00, start: 0.000000, bitrate: N/A Stream #0:0: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 600x300 [SAR 254:254 DAR 2:1], 60 fps, 60 tbr, 60 tbn, 60 tbc Stream mapping: Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264)) Press [q] to stop, [?] for help [libx264 @ 0x14a01e400] using SAR=1/1 [libx264 @ 0x14a01e400] using cpu capabilities: ARMv8 NEON [libx264 @ 0x14a01e400] profile High 4:4:4 Predictive, level 3.1, 4:4:4, 8-bit [libx264 @ 0x14a01e400] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=9 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, matroska, to 'image2.mkv': Metadata: encoder : Lavf58.76.100 Stream #0:0: Video: h264 (H264 / 0x34363248), yuvj444p(pc, bt470bg/unknown/unknown, progressive), 600x300 [SAR 254:254 DAR 2:1], q=2-31, 60 fps, 1k tbn Metadata: encoder : Lavc58.134.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A frame= 60 fps=0.0 q=-1.0 Lsize= 26kB time=00:00:00.95 bitrate= 221.8kbits/s speed=13.7x video:25kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 4.316689% [libx264 @ 0x14a01e400] frame I:1 Avg QP:22.02 size: 415 [libx264 @ 0x14a01e400] frame P:21 Avg QP:21.83 size: 617 [libx264 @ 0x14a01e400] frame B:38 Avg QP:26.79 size: 295 [libx264 @ 0x14a01e400] consecutive B-frames: 6.7% 10.0% 50.0% 33.3% [libx264 @ 0x14a01e400] mb I I16..4: 3.9% 93.9% 2.2% [libx264 @ 0x14a01e400] mb P I16..4: 2.0% 3.2% 1.7% P16..4: 5.9% 3.4% 0.5% 0.0% 0.0% skip:83.3% [libx264 @ 0x14a01e400] mb B I16..4: 1.1% 1.5% 0.3% B16..8: 9.9% 2.0% 0.0% direct: 0.0% skip:85.2% L0:58.0% L1:41.8% BI: 0.2% [libx264 @ 0x14a01e400] 8x8 transform intra:61.5% inter:34.4% [libx264 @ 0x14a01e400] coded y,u,v intra: 5.6% 4.4% 5.2% inter: 0.3% 0.3% 0.4% [libx264 @ 0x14a01e400] i16 v,h,dc,p: 62% 34% 5% 0% [libx264 @ 0x14a01e400] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23% 7% 67% 1% 0% 0% 0% 0% 0% [libx264 @ 0x14a01e400] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 20% 41% 5% 2% 2% 2% 2% 1% [libx264 @ 0x14a01e400] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x14a01e400] ref P L0: 43.2% 11.0% 31.0% 14.9% [libx264 @ 0x14a01e400] ref B L0: 62.0% 28.3% 9.7% [libx264 @ 0x14a01e400] ref B L1: 95.4% 4.6% [libx264 @ 0x14a01e400] kb/s:196.69 Wrote video /var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/image2.mkv Reading frame rate of concat.mkv... r_frame_rate=60/1 avg_frame_rate=60/1 Full reports for each run: /var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/ffprobe-concat.log /var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/ffprobe-image2.log
Here are the two reports from the above run:
comment:8 by , 3 years ago
Replying to Andre LaBranche:
The unwanted 25 fps does seem to be coming from
concat
. I tested that along withimage2
and found thatimage2
honors the frame rate passed as a cli option.
image2 honors the frame_rate
option, but you never gave that option to the image2
instances below the concat demuxer.
I would not oppose a patch to concat to add a time_base
option.
comment:9 by , 22 months ago
Component: | undetermined → avformat |
---|---|
Priority: | normal → wish |
Type: | defect → enhancement |
Version: | unspecified → git-master |
comment:10 by , 6 months ago
Cc: | added |
---|
It seems as if this is due to rounding to a timebase of 1/25 somewhere. The exact timestamps in your example would be 0, 116, 232, 348, 464, 580, 696... which get rounded to 0, 120, 240, 360, 480, 600, 680... when using a timebase of 1/25.