Opened 4 months ago

#11041 new defect

Transcode drops frame that should be truncated by editlist

Reported by: Anne Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: editlist
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:
Came across a case where if the edit-list media-time is specified as in between frame timestamps, that frame will get dropped completely in the output of the transcode. The edit-list media-time is specified as 402. This is in between the 400 and 420 frame timestamps. The expectation is that the frame at 400 is still retained and its duration cut short for the transcode output. However, the frame is dropped completely at this line: https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/mov.c#L4167

The iso documentation has a snippet on edit-lists that seems to backup this interpretation: 'Edits are not restricted to fall on sample times. This means that when entering an edit, it can be necessary to (a) back up to a sync point, and pre-roll from there and then (b) be careful about the duration of the first sample — it might have been truncated if the edit enters it during its normal duration.' It seems like the frame should be truncated if edits are in between sample times.

Safari's player, premiere pro, and QuickTime retain the frame, so this behavior results in inconsistent playback of the original vs the transcoded output, where the output is missing a frame.
How to reproduce/output:

ffmpeg -i original.mov -c:v libx264 -y ffmpeg_output.mov
ffmpeg version 7.0.1 Copyright (c) 2000-2024 the FFmpeg developers
  built with Apple clang version 15.0.0 (clang-1500.3.9.4)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/7.0.1 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags='-Wl,-ld_classic' --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libharfbuzz --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'original.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2024-05-04T18:13:43.000000Z
    com.apple.quicktime.location.accuracy.horizontal: 42.565599
    com.apple.quicktime.location.ISO6709: +47.5327-122.1507+229.837/
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone 11
    com.apple.quicktime.software: 16.2
    com.apple.quicktime.creationdate: 2024-05-04T11:10:39-0700
  Duration: 00:00:12.57, start: 0.000000, bitrate: 8200 kb/s
  Stream #0:0[0x1](und): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, bt709), 1920x1080, 7639 kb/s, 29.15 fps, 29.97 tbr, 600 tbn (default)
      Metadata:
        creation_time   : 2024-05-04T18:13:43.000000Z
        handler_name    : Core Media Video
        vendor_id       : [0][0][0][0]
        encoder         : HEVC
      Side data:
        displaymatrix: rotation of -90.00 degrees
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 145 kb/s (default)
      Metadata:
        creation_time   : 2024-05-04T18:13:43.000000Z
        handler_name    : Core Media Audio
        vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
      Metadata:
        creation_time   : 2024-05-04T18:13:43.000000Z
        handler_name    : Core Media Metadata
  Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
      Metadata:
        creation_time   : 2024-05-04T18:13:43.000000Z
        handler_name    : Core Media Metadata
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0x1226107c0] using cpu capabilities: ARMv8 NEON
[libx264 @ 0x1226107c0] profile High, level 4.0, 4:2:0, 8-bit
[libx264 @ 0x1226107c0] 264 - core 164 r3108 31e19f9 - H.264/MPEG-4 AVC codec - Copyleft 2003-2023 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=24 lookahead_threads=4 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mov, to 'ffmpeg_output.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    com.apple.quicktime.creationdate: 2024-05-04T11:10:39-0700
    com.apple.quicktime.location.accuracy.horizontal: 42.565599
    com.apple.quicktime.location.ISO6709: +47.5327-122.1507+229.837/
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone 11
    com.apple.quicktime.software: 16.2
    encoder         : Lavf61.1.100
  Stream #0:0(und): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1080x1920, q=2-31, 29.97 fps, 30k tbn (default)
      Metadata:
        creation_time   : 2024-05-04T18:13:43.000000Z
        handler_name    : Core Media Video
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.3.100 libx264
      Side data:
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
        displaymatrix: rotation of -0.00 degrees
  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
      Metadata:
        creation_time   : 2024-05-04T18:13:43.000000Z
        handler_name    : Core Media Audio
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.3.100 aac
[out#0/mov @ 0x600001e043c0] video:4606KiB audio:197KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.306282%
frame=  376 fps=183 q=-1.0 Lsize=    4818KiB time=00:00:12.47 bitrate=3162.7kbits/s speed=6.06x    
[libx264 @ 0x1226107c0] frame I:2     Avg QP:21.53  size: 82112
[libx264 @ 0x1226107c0] frame P:99    Avg QP:22.87  size: 25951
[libx264 @ 0x1226107c0] frame B:275   Avg QP:24.07  size:  7210
[libx264 @ 0x1226107c0] consecutive B-frames:  0.5%  3.7%  6.4% 89.4%
[libx264 @ 0x1226107c0] mb I  I16..4: 11.5% 73.7% 14.8%
[libx264 @ 0x1226107c0] mb P  I16..4:  6.4% 13.0%  0.7%  P16..4: 48.5%  6.5%  4.5%  0.0%  0.0%    skip:20.3%
[libx264 @ 0x1226107c0] mb B  I16..4:  0.3%  0.4%  0.0%  B16..8: 40.1%  1.1%  0.1%  direct: 2.1%  skip:55.9%  L0:40.8% L1:57.8% BI: 1.3%
[libx264 @ 0x1226107c0] 8x8 transform intra:64.6% inter:91.7%
[libx264 @ 0x1226107c0] coded y,uvDC,uvAC intra: 27.7% 43.1% 12.0% inter: 10.6% 18.3% 0.2%
[libx264 @ 0x1226107c0] i16 v,h,dc,p: 40% 31% 14% 15%
[libx264 @ 0x1226107c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 18% 51%  2%  3%  3%  4%  2%  2%
[libx264 @ 0x1226107c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 23% 29%  4%  6%  5%  6%  3%  3%
[libx264 @ 0x1226107c0] i8c dc,h,v,p: 56% 23% 18%  3%
[libx264 @ 0x1226107c0] Weighted P-Frames: Y:17.2% UV:6.1%
[libx264 @ 0x1226107c0] ref P L0: 52.6% 26.5% 13.4%  6.8%  0.6%
[libx264 @ 0x1226107c0] ref B L0: 92.4%  5.8%  1.8%
[libx264 @ 0x1226107c0] ref B L1: 97.8%  2.2%
[libx264 @ 0x1226107c0] kb/s:3007.25
[aac @ 0x122684110] Qavg: 234.931

Change History (0)

Note: See TracTickets for help on using tickets.