Opened 4 years ago
Last modified 9 months ago
#9210 new enhancement
Concat images produces wrong timings when duration is in milliseconds
| Reported by: | demet64 | Owned by: | |
|---|---|---|---|
| Priority: | wish | Component: | avformat |
| Version: | git-master | Keywords: | concat |
| Cc: | sycxyc | Blocked By: | |
| Blocking: | Reproduced by developer: | no | |
| Analyzed by developer: | no |
Description (last modified by )
This issue was also mentioned here: https://www.reddit.com/r/ffmpeg/comments/e1o3hv/ffconcat_filter_with_millisecond_precision_how/ and here: https://forum.videohelp.com/threads/397470-Converting-a-sequence-of-images-to-video-with-variable-framerate
Summary of the bug:
How to reproduce:
A concat file like this
file 000000.jpg duration 116ms file 000001.jpg duration 116ms file 000002.jpg duration 116ms file 000003.jpg duration 116ms file 000004.jpg duration 116ms file 000005.jpg duration 116ms file 000006.jpg duration 116ms file 000007.jpg duration 116ms file 000008.jpg duration 116ms file 000009.jpg duration 1002ms file 000010.jpg duration 116ms file 000011.jpg duration 116ms file 000012.jpg duration 116ms file 000013.jpg duration 1129ms file 000014.jpg duration 73ms file 000015.jpg duration 73ms file 000016.jpg duration 116ms file 000017.jpg duration 116ms file 000018.jpg duration 2184ms file 000018.jpg
The output:
ffprobe -f concat -i concat.txt -show_frames | findstr "pkt_pts" pkt_pts=0 pkt_pts_time=0.000000 pkt_pts=3 pkt_pts_time=0.120000 pkt_pts=6 pkt_pts_time=0.240000 pkt_pts=9 pkt_pts_time=0.360000 pkt_pts=12 pkt_pts_time=0.480000 pkt_pts=15 pkt_pts_time=0.600000 pkt_pts=17 pkt_pts_time=0.680000 pkt_pts=20 pkt_pts_time=0.800000 pkt_pts=23 pkt_pts_time=0.920000 pkt_pts=26 pkt_pts_time=1.040000 pkt_pts=51 pkt_pts_time=2.040000 pkt_pts=54 pkt_pts_time=2.160000 pkt_pts=57 pkt_pts_time=2.280000 pkt_pts=60 pkt_pts_time=2.400000 pkt_pts=88 pkt_pts_time=3.520000 pkt_pts=90 pkt_pts_time=3.600000 pkt_pts=92 pkt_pts_time=3.680000 pkt_pts=95 pkt_pts_time=3.800000 pkt_pts=98 pkt_pts_time=3.920000 pkt_pts=152 pkt_pts_time=6.080000
The output looks correct at first, but timestamps are rounded, and for some reasons the duration here
pkt_pts_time=0.600000 pkt_pts=17 pkt_pts_time=0.680000
is 80ms instead of 120ms (and expected 116ms)
It's not the problem of ffprobe, when converting to webm, for example, or gif
ffmpeg -i concat.txt -c:v libvpx-vp9 -lossless 1 out.webm
timings would be like showed above.
The other case:
file 000000.jpg duration 50ms file 000001.jpg duration 60ms file 000002.jpg duration 50ms file 000003.jpg duration 60ms file 000004.jpg duration 50ms file 000005.jpg duration 60ms file 000006.jpg duration 50ms file 000007.jpg duration 60ms file 000008.jpg duration 60ms file 000009.jpg duration 50ms file 000010.jpg duration 60ms file 000011.jpg duration 50ms file 000012.jpg duration 60ms file 000013.jpg duration 50ms file 000014.jpg duration 60ms file 000015.jpg duration 50ms file 000016.jpg duration 60ms file 000017.jpg duration 60ms file 000018.jpg duration 50ms file 000019.jpg duration 60ms file 000020.jpg duration 50ms file 000021.jpg duration 60ms file 000022.jpg duration 50ms file 000023.jpg duration 60ms file 000023.jpg
Output:
pkt_pts=0 pkt_pts_time=0.000000 pkt_pts=1 pkt_pts_time=0.040000 pkt_pts=3 pkt_pts_time=0.120000 pkt_pts=4 pkt_pts_time=0.160000 pkt_pts=6 pkt_pts_time=0.240000 pkt_pts=7 pkt_pts_time=0.280000 pkt_pts=8 pkt_pts_time=0.320000 pkt_pts=10 pkt_pts_time=0.400000 pkt_pts=11 pkt_pts_time=0.440000 pkt_pts=13 pkt_pts_time=0.520000 pkt_pts=14 pkt_pts_time=0.560000 pkt_pts=15 pkt_pts_time=0.600000 pkt_pts=17 pkt_pts_time=0.680000 pkt_pts=18 pkt_pts_time=0.720000 pkt_pts=19 pkt_pts_time=0.760000 pkt_pts=21 pkt_pts_time=0.840000 pkt_pts=22 pkt_pts_time=0.880000 pkt_pts=24 pkt_pts_time=0.960000 pkt_pts=25 pkt_pts_time=1.000000 pkt_pts=26 pkt_pts_time=1.040000 pkt_pts=28 pkt_pts_time=1.120000 pkt_pts=29 pkt_pts_time=1.160000 pkt_pts=31 pkt_pts_time=1.240000 pkt_pts=32 pkt_pts_time=1.280000 pkt_pts=33 pkt_pts_time=1.320000
Attachments (2)
Change History (12)
comment:1 by , 4 years ago
| Description: | modified (diff) |
|---|
comment:2 by , 4 years ago
comment:3 by , 4 years ago
It seems as if this is due to rounding to a timebase of 1/25 somewhere.
This is what person from reddit thread I mentioned says too. The problem is that it happens on input stream and it seems there's no way you can specify timebase of the input stream
comment:4 by , 4 years ago
Please (test current FFmpeg git head and) provide the command line you tested together with the complete, uncut console output and the necessary input files to make this a valid ticket.
comment:5 by , 4 years ago
| Keywords: | concat added |
|---|
by , 4 years ago
| Attachment: | ffmpeg-20210501-201727.log added |
|---|
by , 4 years ago
| Attachment: | concat.txt added |
|---|
comment:6 by , 4 years ago
I don't know if input images are necessary. I tried the same concat file with different sets of pictures and it didn't have any impact on timecodes.
follow-up: 8 comment:7 by , 3 years ago
The unwanted 25 fps does seem to be coming from concat. I tested that along with image2 and found that image2 honors the frame rate passed as a cli option.
Here's a script that creates some sample images, does a concat run and an image2 run, and collects reports for each.
#!/bin/zsh
OUT=$(mktemp -d)
CATFILE=${OUT}/catfile.txt
pushd ${OUT}
echo "generating some sample images with Imagemagik's 'convert' tool"
for i in {1..60}
do
IMG=image-$(printf "%03d" ${i}).jpg
convert -size 600x300 -gravity center -background black \
-fill purple label:${i} -units PixelsPerInch -density 254 ${IMG}
echo "file '${IMG}'" >> ${CATFILE}
echo "duration 0.016" >> ${CATFILE}
done
echo "Wrote sample ffmpeg concat input file: ${CATFILE}"
CONCAT_CMD='ffmpeg -f concat -i ${CATFILE} -report concat.mkv > /dev/null'
export FFREPORT=file=ffprobe-concat.log
echo "Probing concat with:\n${CONCAT_CMD}"
eval ${CONCAT_CMD}
echo "Wrote video ${OUT}/concat.mkv"
echo "Reading frame rate of concat.mkv..."
unset FFREPORT
ffprobe -v error -show_format -show_streams concat.mkv | grep 'frame_rate'
IMAGE2_CMD="ffmpeg -f image2 -framerate 60 -i image-%03d.jpg -report image2.mkv > /dev/null"
echo "\nProbing image2 with:\n${IMAGE2_CMD}"
export FFREPORT=file=ffprobe-image2.log
eval ${IMAGE2_CMD}
echo "Wrote video ${OUT}/image2.mkv"
echo "Reading frame rate of concat.mkv..."
unset FFREPORT
ffprobe -v error -show_format -show_streams image2.mkv | grep 'frame_rate'
echo "\nFull reports for each run: "
ls -1 ${OUT}/ffprobe*.log
open ${OUT}
popd
When run on my machine, it looks like this:
(base) andre@boom matlab_image_tsp % ./duration_snooping.sh
generating some sample images with Imagemagik's 'convert' tool
Wrote sample ffmpeg concat input file: /var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/catfile.txt
Probing concat with:
ffmpeg -f concat -i ${CATFILE} -report concat.mkv > /dev/null
ffmpeg started on 2021-12-06 at 04:07:22
Report written to "ffprobe-concat.log"
Log level: 48
ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers
built with Apple clang version 13.0.0 (clang-1300.0.29.3)
configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/4.4.1_3 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-avresample --enable-videotoolbox
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
Input #0, concat, from '/var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/catfile.txt':
Duration: 00:00:00.96, start: 0.000000, bitrate: 18 kb/s
Stream #0:0: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 600x300 [SAR 254:254 DAR 2:1], 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x12b815400] using SAR=1/1
[libx264 @ 0x12b815400] using cpu capabilities: ARMv8 NEON
[libx264 @ 0x12b815400] profile High 4:4:4 Predictive, level 2.1, 4:4:4, 8-bit
[libx264 @ 0x12b815400] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=9 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'concat.mkv':
Metadata:
encoder : Lavf58.76.100
Stream #0:0: Video: h264 (H264 / 0x34363248), yuvj444p(pc, bt470bg/unknown/unknown, progressive), 600x300 [SAR 254:254 DAR 2:1], q=2-31, 25 fps, 1k tbn
Metadata:
encoder : Lavc58.134.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame= 26 fps=0.0 q=-1.0 Lsize= 14kB time=00:00:00.92 bitrate= 121.6kbits/s dup=0 drop=34 speed=13.4x
video:13kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 6.463387%
[libx264 @ 0x12b815400] frame I:1 Avg QP:19.46 size: 495
[libx264 @ 0x12b815400] frame P:12 Avg QP:17.97 size: 657
[libx264 @ 0x12b815400] frame B:13 Avg QP:22.41 size: 314
[libx264 @ 0x12b815400] consecutive B-frames: 11.5% 61.5% 11.5% 15.4%
[libx264 @ 0x12b815400] mb I I16..4: 4.2% 93.6% 2.2%
[libx264 @ 0x12b815400] mb P I16..4: 1.6% 2.6% 1.4% P16..4: 8.6% 3.2% 0.5% 0.0% 0.0% skip:82.1%
[libx264 @ 0x12b815400] mb B I16..4: 0.8% 1.3% 0.3% B16..8: 9.7% 2.8% 0.1% direct: 0.0% skip:84.9% L0:38.7% L1:60.7% BI: 0.7%
[libx264 @ 0x12b815400] 8x8 transform intra:71.4% inter:27.3%
[libx264 @ 0x12b815400] coded y,u,v intra: 5.0% 4.1% 4.5% inter: 0.5% 0.3% 0.4%
[libx264 @ 0x12b815400] i16 v,h,dc,p: 53% 38% 8% 0%
[libx264 @ 0x12b815400] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 31% 9% 58% 1% 0% 0% 0% 0% 0%
[libx264 @ 0x12b815400] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 21% 47% 3% 2% 2% 2% 1% 2%
[libx264 @ 0x12b815400] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x12b815400] ref P L0: 30.0% 7.6% 51.8% 10.6%
[libx264 @ 0x12b815400] ref B L0: 75.2% 24.0% 0.7%
[libx264 @ 0x12b815400] ref B L1: 96.8% 3.2%
[libx264 @ 0x12b815400] kb/s:95.87
Wrote video /var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/concat.mkv
Reading frame rate of concat.mkv...
r_frame_rate=25/1
avg_frame_rate=25/1
Probing image2 with:
ffmpeg -f image2 -framerate 60 -i image-%03d.jpg -report image2.mkv > /dev/null
ffmpeg started on 2021-12-06 at 04:07:22
Report written to "ffprobe-image2.log"
Log level: 48
ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers
built with Apple clang version 13.0.0 (clang-1300.0.29.3)
configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/4.4.1_3 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-avresample --enable-videotoolbox
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
Input #0, image2, from 'image-%03d.jpg':
Duration: 00:00:01.00, start: 0.000000, bitrate: N/A
Stream #0:0: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 600x300 [SAR 254:254 DAR 2:1], 60 fps, 60 tbr, 60 tbn, 60 tbc
Stream mapping:
Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x14a01e400] using SAR=1/1
[libx264 @ 0x14a01e400] using cpu capabilities: ARMv8 NEON
[libx264 @ 0x14a01e400] profile High 4:4:4 Predictive, level 3.1, 4:4:4, 8-bit
[libx264 @ 0x14a01e400] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=9 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'image2.mkv':
Metadata:
encoder : Lavf58.76.100
Stream #0:0: Video: h264 (H264 / 0x34363248), yuvj444p(pc, bt470bg/unknown/unknown, progressive), 600x300 [SAR 254:254 DAR 2:1], q=2-31, 60 fps, 1k tbn
Metadata:
encoder : Lavc58.134.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame= 60 fps=0.0 q=-1.0 Lsize= 26kB time=00:00:00.95 bitrate= 221.8kbits/s speed=13.7x
video:25kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 4.316689%
[libx264 @ 0x14a01e400] frame I:1 Avg QP:22.02 size: 415
[libx264 @ 0x14a01e400] frame P:21 Avg QP:21.83 size: 617
[libx264 @ 0x14a01e400] frame B:38 Avg QP:26.79 size: 295
[libx264 @ 0x14a01e400] consecutive B-frames: 6.7% 10.0% 50.0% 33.3%
[libx264 @ 0x14a01e400] mb I I16..4: 3.9% 93.9% 2.2%
[libx264 @ 0x14a01e400] mb P I16..4: 2.0% 3.2% 1.7% P16..4: 5.9% 3.4% 0.5% 0.0% 0.0% skip:83.3%
[libx264 @ 0x14a01e400] mb B I16..4: 1.1% 1.5% 0.3% B16..8: 9.9% 2.0% 0.0% direct: 0.0% skip:85.2% L0:58.0% L1:41.8% BI: 0.2%
[libx264 @ 0x14a01e400] 8x8 transform intra:61.5% inter:34.4%
[libx264 @ 0x14a01e400] coded y,u,v intra: 5.6% 4.4% 5.2% inter: 0.3% 0.3% 0.4%
[libx264 @ 0x14a01e400] i16 v,h,dc,p: 62% 34% 5% 0%
[libx264 @ 0x14a01e400] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23% 7% 67% 1% 0% 0% 0% 0% 0%
[libx264 @ 0x14a01e400] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 20% 41% 5% 2% 2% 2% 2% 1%
[libx264 @ 0x14a01e400] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x14a01e400] ref P L0: 43.2% 11.0% 31.0% 14.9%
[libx264 @ 0x14a01e400] ref B L0: 62.0% 28.3% 9.7%
[libx264 @ 0x14a01e400] ref B L1: 95.4% 4.6%
[libx264 @ 0x14a01e400] kb/s:196.69
Wrote video /var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/image2.mkv
Reading frame rate of concat.mkv...
r_frame_rate=60/1
avg_frame_rate=60/1
Full reports for each run:
/var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/ffprobe-concat.log
/var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/ffprobe-image2.log
Here are the two reports from the above run:
comment:8 by , 3 years ago
Replying to Andre LaBranche:
The unwanted 25 fps does seem to be coming from
concat. I tested that along withimage2and found thatimage2honors the frame rate passed as a cli option.
image2 honors the frame_rate option, but you never gave that option to the image2 instances below the concat demuxer.
I would not oppose a patch to concat to add a time_base option.
comment:9 by , 2 years ago
| Component: | undetermined → avformat |
|---|---|
| Priority: | normal → wish |
| Type: | defect → enhancement |
| Version: | unspecified → git-master |
comment:10 by , 9 months ago
| Cc: | added |
|---|
Note:
See TracTickets
for help on using tickets.
It seems as if this is due to rounding to a timebase of 1/25 somewhere. The exact timestamps in your example would be 0, 116, 232, 348, 464, 580, 696... which get rounded to 0, 120, 240, 360, 480, 600, 680... when using a timebase of 1/25.