Opened 13 months ago

Closed 2 months ago

#10255 closed defect (fixed)

prores_ks encoder on Mac with M2 creates flickering

Reported by: jonnyburger Owned by:
Priority: normal Component: avcodec
Version: unspecified Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description (last modified by jonnyburger)

Summary of the bug:

On an M2 Mac, encoding an image sequence with -profile:v 4 (4444) and the pixel format yuva444p10le creates a flickering video.

The issue was reproduced by two separated developers, but the bug does not happen on a M1 Mac. We suspect it has to do with the Apple Silicon Media Engine.

FFmpeg version: N-110004-g1231003c3c (build from latest development commit)

How to reproduce:

Get the reproduction from https://streams.videolan.org/ffmpeg/incoming/prores-repro.zip

Command

ffmpeg \
-r 30 \
-f image2 \
-s 1280x720 \
-start_number 0 \
-i out/element-%03d.png \
-vcodec prores_ks \
-profile:v 4 \
-pix_fmt yuva444p10le \
-map_metadata -1 \
-vendor apl0 \
-metadata comment="Made with Remotion 3.3.75" \
-y \
out.mov

Change History (12)

comment:1 by jonnyburger, 13 months ago

Description: modified (diff)

comment:2 by Thomas, 7 months ago

I'd like to add something: the problem is not when encoding on an M2 Mac on my end, but when decoding on an M2 Mac, which is a bigger problem since I can't solve this problem for M2 users when I encode a prores (with alpha) video on a Linux machine.

comment:3 by Elon Musk, 7 months ago

So to clarify and to be able to reproduce, check which one/multiply are correct/true:

  • decoding is not correct
  • encoding is not correct
  • decoding is not working properly on certain arch combo
  • encoding is not working properly on certain arch combo
  • when decoding always native ffmpeg decoder is used?

comment:4 by Thomas, 7 months ago

Sorry about the misunderstanding.
Actually, this is the prores_ks encoding (4444 w/alpha) that is not correct, on all platforms.

Flickering only occurs when playing the video on a Mac M2 though, using native players such as Quicktime, iMovie, etc.

To reproduce:

  • encode a prores 4444 w/alpha video on any platform
  • play the video on a Mac M2 using Quicktime

comment:5 by jonnyburger, 7 months ago

I agree with Thomas and the original bug report is unprecise.

The encoding is the same on both M1 and M2, but the playback on M2 is wrong.

To complete the reproduction: Take the reproduction from the first post, and play the video with QuickTime on macOS on M2.

To answer your question:

  • decoding on macOS does not work correctly when the video is encoded with FFmpeg
  • the QuickTime player is used for decoding
  • It is not clear yet whether the fault is in the encoder or decoder
  • The decoding works correctly on M1 arch, but not on M2 arch
  • The encoding is presumably the same on both platforms, since it is software encoding.

My company remotion.dev is paying out a $500 bounty for the developer who can submit a patch!

This fix has value to us because it would allow users to export transparent videos and drag them into other video editing programs.

The bounty can be claimed by emailing jonny@remotion.dev.

in reply to:  3 comment:6 by Kieran Kunhya, 7 months ago

Replying to Elon Musk:

So to clarify and to be able to reproduce, check which one/multiply are correct/true:

  • decoding is not correct
  • encoding is not correct
  • decoding is not working properly on certain arch combo
  • encoding is not working properly on certain arch combo
  • when decoding always native ffmpeg decoder is used?

FFmpeg also has videotoolbox support for hardware decoding of ProRes so this should be checked too. Likely Quicktime is using this same decode API but I don't know for sure.

Last edited 7 months ago by Kieran Kunhya (previous) (diff)

comment:7 by jonnyburger, 7 months ago

I tried on a M2 Mac on a copy of FFmpeg (N-111888-gfc993e7a53), enabled with --enable-videotoolbox, as an additional step of my repro:

ffmpeg -hwaccel videotoolbox -i out.mov out.mp4 -y

It seems to have picked up, since values other than videotoolbox throw an error.

The video transcodes without any flickering, although quality is degraded.

I used ffmpeg to convert a video, because according of my knowledge, ffplay does not support the hwaccel flag.

comment:8 by Clément Bœsch, 4 months ago

I'm looking into the issue currently.

comment:9 by jonnyburger, 3 months ago

By the way, we (remotion.dev) have put out a $500 bounty to anyone can solve this: https://twitter.com/Remotion/status/1697141747426492422

comment:10 by Clément Bœsch, 3 months ago

The following pending patch should address the issue: https://ffmpeg.org//pipermail/ffmpeg-devel/2023-December/318922.html

comment:11 by jonnyburger, 3 months ago

For us, the patch solves the issue!

comment:12 by Marth64, 2 months ago

Resolution: fixed
Status: newclosed

Closing because patch was merged and user indicates it resolved the issue. Thank you.

Note: See TracTickets for help on using tickets.