Opened 12 months ago

Closed 5 months ago

#7163 closed enhancement (fixed)

12-bit ProRes not supported

Reported by: spookybathtub Owned by:
Priority: wish Component: avcodec
Version: git-master Keywords: prores
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:
Apple Prores 444 and 444 XQ can be either 10-bit or 12-bit, but I believe ffmpeg always decodes as 10-bit. For example, this console output shows the image as yuv444p10le. Arri claims that this file is 12-bit. Sample files such as this can be downloaded here: http://www.arri.com/camera/alexa/learn/alexa_sample_footage

In source code proresdsp.h, I see this line:
#define PRORES_BITS_PER_SAMPLE 10 ///< output precision of prores decoder

https://images.apple.com/support/finalcutpro/docs/Apple-ProRes-White-Paper-July-2009.pdf

$ ffmpeg -i /Scratch/sample\ footage/Alexa\ LogC/M001C007_161207_R00H.mov 
ffmpeg version 3.4.git Copyright (c) 2000-2018 the FFmpeg developers
  built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
  configuration: --prefix=/opt/local --enable-swscale --enable-avfilter --enable-avresample --enable-libmp3lame --enable-libvorbis --enable-libopus --enable-librsvg --enable-libtheora --enable-libopenjpeg --enable-libmodplug --enable-libvpx --enable-libsoxr --enable-libspeex --enable-libass --enable-libbluray --enable-lzma --enable-gnutls --enable-fontconfig --enable-libfreetype --enable-libfribidi --disable-libjack --disable-libopencore-amrnb --disable-libopencore-amrwb --disable-indev=jack --enable-opencl --disable-outdev=xv --enable-audiotoolbox --enable-videotoolbox --enable-sdl2 --mandir=/opt/local/share/man --enable-shared --enable-pthreads --cc=/usr/bin/clang --arch=x86_64 --enable-x86asm --enable-libx265 --enable-gpl --enable-postproc --enable-libx264 --enable-libxvid --enable-nonfree --enable-libfdk-aac
  libavutil      56. 12.100 / 56. 12.100
  libavcodec     58. 15.100 / 58. 15.100
  libavformat    58. 10.100 / 58. 10.100
  libavdevice    58.  2.100 / 58.  2.100
  libavfilter     7. 13.100 /  7. 13.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  0.102 /  5.  0.102
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Guessed Channel Layout for Input Stream #0.1 : 5.0
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Scratch/sample footage/Alexa LogC/M001C007_161207_R00H.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 537199360
    compatible_brands: qt  ARRI
    creation_time   : 2016-12-07T15:18:41.000000Z
    com.apple.finalcutstudio.media.uuid: 4AC6826A-0538-4BCC-A0B0-11F520AC11C9
    com.arri.camera.CameraId: R00H
    com.arri.camera.CameraIndex: M
    com.arri.camera.CameraModel: ARRI ALEXA Mini
    com.arri.camera.CameraSerialNumber: 20017
    com.arri.camera.ColorGammaSxS: LOG-C
    com.arri.camera.ExposureIndexAsa: 800
    com.arri.camera.LookFileBurnedIn: No
    com.arri.camera.ProductionInfoOperator: GHI
    com.arri.camera.NdFilterDensity: 1200
    com.arri.camera.NdFilterType: 1
    com.arri.camera.ProductionInfoCinematographer: DEF
    com.arri.camera.ProductionInfoDirector: ABC
    com.arri.camera.ProductionInfoLocation: BOTMUC
    com.arri.camera.ProductionInfoProduction: ARRI
    com.arri.camera.ProductionInfoUser1: 
    com.arri.camera.ProductionInfoUser2: 
    com.arri.camera.ProjectFps: 24000
    com.arri.camera.ReelName: M001R00H
    com.arri.camera.SensorFps: 24000
    com.arri.camera.ShutterAngle: 1728
    com.arri.camera.SoundReel: 
    com.arri.camera.SupVersion: 4.02.05
    com.arri.camera.SxsSerialNumber: 141101300018
    com.arri.camera.UserDate: 20161207
    com.arri.camera.UserTime: 15h18m41
    com.arri.camera.WhiteBalanceKelvin: 5600
    com.arri.camera.WhiteBalanceTintCc: 0
    com.arri.camera.CameraClipName: M001C007_161207_R00H.mov
    com.arri.camera.Product: 2
    com.arri.camera.SubProduct: 1
    com.arri.camera.look.name: ARRI 709.AML
    com.arri.camera.look.user_lut: 0
    com.apple.proapps.color.asc-cdl: 
    com.arri.camera.look.lut3d: 
    com.arri.camera.look.lut3d_with_cdl: 
    com.arri.camera.look.video_param_with_target_colorspace: 
    com.arri.camera.ProductionInfoCompany: ARRITEST
    com.arri.camera.SceneName: 
    com.arri.camera.TakeName: 
    com.arri.camera.audio.Configuration: 
    com.arri.camera.LensSerialNumber: 0
    com.arri.camera.LensType: 
    com.arri.camera.UnitPreference: Metric
    com.arri.camera.WbTracking: 0
    com.arri.camera.ImageOrientation: 0
    com.arri.camera.ImageSharpness: 0
    com.arri.camera.ImageDetail: 0
    com.arri.camera.ImageDenoising: 0
    com.arri.camera.DynamicMetadataVersion: 65536
    com.arri.camera.audio.BluetoothEnabled: 0
    com.arri.camera.sensor.PhotoSites: 2560x2145
    com.arri.camera.PixelAspectRatio: 0.000000
    com.arri.camera.FramelineFileName1: ARRI 6by5 1.78 Scope 2x
    com.arri.camera.FramelineRect1A.Name: 1.78:1anamorphic2x_scaling100%
    com.arri.camera.FramelineRect1A.Left: 261
    com.arri.camera.FramelineRect1A.Top: 0
    com.arri.camera.FramelineRect1A.Width: 1525
    com.arri.camera.FramelineRect1A.Height: 858
    com.arri.camera.FramelineRect1A.Type: 1
    timecode        : 01:55:44:01
  Duration: 00:00:10.13, start: 0.000000, bitrate: 283348 kb/s
    Stream #0:0(eng): Video: prores (ap4x / 0x78347061), yuv444p10le(bt709, progressive), 2048x858, 271534 kb/s, SAR 1:1 DAR 1024:429, 24 fps, 24 tbr, 24 tbn, 24 tbc (default)
    Metadata:
      creation_time   : 2016-12-07T15:18:41.000000Z
      handler_name    : ?Apple Alias Data Handler
      encoder         : Apple ProRes 4444 (XQ)
    Stream #0:1(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, 5.0, s32 (24 bit), 5760 kb/s (default)
    Metadata:
      creation_time   : 2016-12-07T15:18:41.000000Z
      handler_name    : ?Apple Alias Data Handler
    Stream #0:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
    Metadata:
      creation_time   : 2016-12-07T15:18:41.000000Z
      handler_name    : ?Apple Alias Data Handler
      reel_name       : M001R00H
      timecode        : 01:55:44:01
At least one output file must be specified

Attachments (7)

DPX12_fromResolve_stretched.png (45.9 KB) - added by spookybathtub 12 months ago.
PR422_fromResolve_stretched.png (38.7 KB) - added by spookybathtub 12 months ago.
PR444_fromResolve_stretched.png (52.0 KB) - added by spookybathtub 12 months ago.
PR444_Resolve_ffmpeg_TIFF16_stretched.png (37.6 KB) - added by spookybathtub 12 months ago.
TIFF16_original_stretched.png (46.0 KB) - added by spookybathtub 12 months ago.
ResolvePR444full.mov (46.1 KB) - added by spookybathtub 12 months ago.
SMPTE ProRes rdd36-2015.pdf (223.9 KB) - added by spookybathtub 11 months ago.
ProRes? Bitstream Syntax and Decoding Process

Download all attachments as: .zip

Change History (16)

comment:1 Changed 12 months ago by cehoyos

  • Keywords prores added
  • Priority changed from normal to wish
  • Resolution set to needs_more_info
  • Status changed from new to closed

Marketing documents are not helpful in this context.

Changed 12 months ago by spookybathtub

Changed 12 months ago by spookybathtub

Changed 12 months ago by spookybathtub

Changed 12 months ago by spookybathtub

Changed 12 months ago by spookybathtub

Changed 12 months ago by spookybathtub

comment:2 Changed 12 months ago by spookybathtub

Here is my test. I make a 16-bit TIFF in Photoshop with a gray ramp from RGB value 12850 to 15420. This makes the result easier to see than doing a full black to white ramp. This is 2570 code values in an image 1920 pixels wide, so it looks perfectly smooth on a waveform.

I import this TIFF to Resolve, and render three ways: Prores 444, Prores 422, and DPX 12 bit RGB, all using full-range levels. Then I bring those renders back into Resolve and I stretch the contrast out so this gray section fills the whole waveform.

Now in the Prores 422, I count 80 code values in the waveform, which is exactly what I would expect for 10-bit precision. The PR444 and the DPX look nearly identical. I expect there to be 320 steps, but I’m not going to count them all. The PR444 looks a bit smoother due to dithering. This proves that the Prores 444 is 12-bit.

With ffmpeg, I convert the PR444 to 16-bit TIFF. I bring this TIFF into Resolve, and stretch the contrast, and here I count 80 steps in the waveform. This proves that ffmpeg is decoding the PR444 using 10-bit precision.

I have attached screenshots of each waveform. The original TIFF is too large to upload here, but I included the PR444 rendered from Resolve, which you can use to replicate my test.

$ ffmpeg -i ResolvePR444full.mov -pix_fmt rgb48le PR444full_ffmpeg16b.tif
ffmpeg version 3.4.git Copyright (c) 2000-2018 the FFmpeg developers
  built with Apple LLVM version 9.1.0 (clang-902.0.39.1)
  configuration: --prefix=/opt/local --enable-swscale --enable-avfilter --enable-avresample --enable-libmp3lame --enable-libvorbis --enable-libopus --enable-librsvg --enable-libtheora --enable-libopenjpeg --enable-libmodplug --enable-libvpx --enable-libsoxr --enable-libspeex --enable-libass --enable-libbluray --enable-lzma --enable-gnutls --enable-fontconfig --enable-libfreetype --enable-libfribidi --disable-libjack --disable-libopencore-amrnb --disable-libopencore-amrwb --disable-libxcb --disable-libxcb-shm --disable-libxcb-xfixes --disable-indev=jack --enable-opencl --disable-outdev=xv --enable-audiotoolbox --enable-videotoolbox --enable-sdl2 --mandir=/opt/local/share/man --enable-shared --enable-pthreads --cc=/usr/bin/clang --arch=x86_64 --enable-x86asm --enable-libx265 --enable-gpl --enable-postproc --enable-libx264 --enable-libxvid --enable-nonfree --enable-libfdk-aac
  libavutil      56. 12.100 / 56. 12.100
  libavcodec     58. 15.100 / 58. 15.100
  libavformat    58. 10.100 / 58. 10.100
  libavdevice    58.  2.100 / 58.  2.100
  libavfilter     7. 13.100 /  7. 13.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  0.102 /  5.  0.102
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'ResolvePR444full.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2018-04-25T22:00:22.000000Z
  Duration: 00:00:00.04, start: 0.000000, bitrate: 9062 kb/s
    Stream #0:0(und): Video: prores (ap4h / 0x68347061), yuv444p10le(bt709/bt709/bt470m, progressive), 1920x1080, 8788 kb/s, SAR 1:1 DAR 16:9, 24 fps, 24 tbr, 24 tbn, 24 tbc (default)
    Metadata:
      creation_time   : 2018-04-25T22:00:22.000000Z
      handler_name    : Core Media Data Handler
      encoder         : Apple ProRes 4444
      timecode        : 01:00:01:04
    Stream #0:1(und): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
    Metadata:
      creation_time   : 2018-04-25T22:00:22.000000Z
      handler_name    : Core Media Data Handler
      timecode        : 01:00:01:04
File 'PR444full_ffmpeg16b.tif' already exists. Overwrite ? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (prores (native) -> tiff (native))
Press [q] to stop, [?] for help
Output #0, image2, to 'PR444full_ffmpeg16b.tif':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    encoder         : Lavf58.10.100
    Stream #0:0(und): Video: tiff, rgb48le, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 24 fps, 24 tbn, 24 tbc (default)
    Metadata:
      creation_time   : 2018-04-25T22:00:22.000000Z
      handler_name    : Core Media Data Handler
      timecode        : 01:00:01:04
      encoder         : Lavc58.15.100 tiff
frame=    1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A speed=0.352x    
video:12129kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

comment:3 Changed 12 months ago by spookybathtub

  • Resolution needs_more_info deleted
  • Status changed from closed to reopened

Changed 11 months ago by spookybathtub

ProRes? Bitstream Syntax and Decoding Process

comment:4 Changed 11 months ago by spookybathtub

Apparently it's impossible to detect the bit depth of a ProRes file, so we should have a command-line option to switch between 10 and 12. It would make sense to default to 10 for 4:2:2 sources and 12 for 4:4:4 sources, since that is how these formats are commonly used.

comment:5 Changed 5 months ago by kieranjol

So the ffmpeg-user thread is no longer active, but does it look like from this testing by spookybathtub that these really are 12-bit files?

comment:6 Changed 5 months ago by richardpl

They are 12bit, otherwise bug would be already closed.

comment:7 Changed 5 months ago by kierank

Here is unfinished patch, need to add 12-bit alpha:
https://pastebin.com/mYNJkdMH

comment:8 Changed 5 months ago by spookybathtub

Thanks for your work on this. Since there is no way to identify Prores bit depth by reading a metadata flag, I suggest to always decode Prores 444 as 12-bit by default, unless the user chooses 10-bit with a command-line flag to improve speed. Prores 422 can remain as 10-bit.
Keep in mind the alpha channel should be 16 bits. Although I have personally never seen a Prores file with alpha.

comment:9 Changed 5 months ago by cehoyos

  • Resolution set to fixed
  • Status changed from reopened to closed
  • Version changed from unspecified to git-master

Fixed by Martin Vignali and Kieran Kunhya in ffafa53dbf06c8967aada9b7c0fc7ef9cb374fc0
Thank you for the particularly useful report!

Note: See TracTickets for help on using tickets.