displaymatrix side data carried through to output when overlay filter is used

[Wes Castro]

Summary of the bug:

When using an input with rotation metadata (displaymatrix side data) with the overlay filter, the rotation metadata will be carried through to the encoded output. Note that the pixels of the input will have already been "physically" rotated, effectively consuming the rotation metadata. If a transcode is made without the overlay filter the pixels are still rotated but the rotation metadata is dropped as expected.

This has happened since at least FFmpeg 7.1 and on the master branch (bb87d19cd9720631d79e67a5ecca4659ae97734a). With FFmpeg 7.0 the issue doesn't reproduce (displaymatrix sidedata is dropped when the overlay filter is used)

How to reproduce:

• Generate rotated test video (1920x1080, rotated 90degs to 1080x1920 during playback or transcoding) and test image

  ffmpeg -y -display_rotation 90 -noautorotate -f lavfi -i smptebars=size=1920x1080:rate=30:duration=5 -pix_fmt yuv420p -c:v libx264 -crf 12 ~/rotated_video.mp4
  ffmpeg -y -f lavfi -i smptebars=size=128x128:rate=1:duration=1  ~/image.png
  

• Generate transcode with overlay filter

  ffmpeg -y -i ~/rotated_video.mp4 -f image2 -framerate 30 -i ~/image.png -filter_complex "[0:v][1:v] overlay" ~/output1.mp4
  

FFprobe output of ~/output1.mp4:

  Stream #0:0[0x1](und): Video: mpeg4 (Simple Profile) (mp4v / 0x7634706D), yuv420p, 1080x1920 [SAR 1:1 DAR 9:16], 1433 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
      Metadata:
        handler_name    : VideoHandler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.33.102 mpeg4
      Side data:
        displaymatrix: rotation of 90.00 degrees

^{Note the displaymatrix side data}

• Generate transcode without overlay filter

  ffmpeg -y -i ~/rotated_video ~/output2.mp4
  

FFprobe output of ~/output2.mp4:

  Stream #0:0[0x1](und): Video: mpeg4 (Simple Profile) (mp4v / 0x7634706D), yuv420p, 1080x1920 [SAR 1:1 DAR 9:16], 1358 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
      Metadata:
        handler_name    : VideoHandler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.33.102 mpeg4

^{Note there is no displaymatrix side data}

[James]

Fixed in 01f63ef0b44fa70d87edaf52f19fcdb2ae7446b3