dnxhdenc.c forces DNxHR HQX to 10-bit, but it should allow 12-bit

[electron.rotoscope]

According to the Avid specifications for the DNxHR HQX and DNxHR 444 formats, they should always be 12-bit as shown on ​http://avid.force.com/pkb/articles/en_US/White_Paper/DNxHR-Codec-Bandwidth-Specifications in the section "DNxHR Key"

The current version of ffmpeg forces the output to 10-bit. This is referenced (as far as I can tell) in lines 385 to 398 of libavcodec\dnxhdenc.c

    if ((ctx->profile == FF_PROFILE_DNXHR_444 && (avctx->pix_fmt != AV_PIX_FMT_YUV444P10 &&
                                                  avctx->pix_fmt != AV_PIX_FMT_GBRP10)) ||
        (ctx->profile != FF_PROFILE_DNXHR_444 && (avctx->pix_fmt == AV_PIX_FMT_YUV444P10 ||
                                                  avctx->pix_fmt == AV_PIX_FMT_GBRP10))) {
        av_log(avctx, AV_LOG_ERROR,
               "pixel format is incompatible with DNxHD profile\n");
        return AVERROR(EINVAL);
    }

    if (ctx->profile == FF_PROFILE_DNXHR_HQX && avctx->pix_fmt != AV_PIX_FMT_YUV422P10) {
        av_log(avctx, AV_LOG_ERROR,
               "pixel format is incompatible with DNxHR HQX profile\n");
        return AVERROR(EINVAL);
    }

It looks like this may have been discussed in 2016 but maybe never implemented, not sure https://ffmpeg.org/pipermail/ffmpeg-devel/2016-August/197897.html

[electron.rotoscope]

How to reproduce (using an Avid-generated HQX file as an input)

D:\ffmpeg.exe -i "D:\HQX_Bars.mov" -c:v dnxhd -profile:v dnxhr_hqx -pix_fmt yuv422p12le -an "X:\output.mov"
ffmpeg version N-91330-ga990184007 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7.3.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
  libavutil      56. 18.102 / 56. 18.102
  libavcodec     58. 20.103 / 58. 20.103
  libavformat    58. 17.100 / 58. 17.100
  libavdevice    58.  4.101 / 58.  4.101
  libavfilter     7. 25.100 /  7. 25.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'D:\HQX_Bars.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 537199360
    compatible_brands: qt
    creation_time   : 2018-06-26T21:22:27.000000Z
    timecode        : 01:00:00:00
  Duration: 00:00:01.00, start: 0.000000, bitrate: 698462 kb/s
    Stream #0:0(eng): Video: dnxhd (DNXHR HQX) (AVdh / 0x68645641), yuv422p12le(pc, bt709/unknown/unknown), 3840x2160, 698438 kb/s, 23.98 fps, 23.98 tbr, 23976 tbn, 23976 tbc (default)
    Metadata:
      creation_time   : 2018-06-26T21:22:27.000000Z
      handler_name    : Apple Alias Data Handler
      encoder         : Avid DNxHR Codec
    Stream #0:1(eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
    Metadata:
      creation_time   : 2018-06-26T21:22:27.000000Z
      handler_name    : Apple Alias Data Handler
      timecode        : 01:00:00:00
Stream mapping:
  Stream #0:0 -> #0:0 (dnxhd (native) -> dnxhd (native))
Press [q] to stop, [?] for help
Incompatible pixel format 'yuv422p12le' for codec 'dnxhd', auto-selecting format 'yuv422p10le'
Output #0, mov, to 'X:\output.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 537199360
    compatible_brands: qt
    timecode        : 01:00:00:00
    encoder         : Lavf58.17.100
    Stream #0:0(eng): Video: dnxhd (DNXHR HQX) (AVdh / 0x68645641), yuv422p10le, 3840x2160, q=2-1024, 200 kb/s, 0.04 fps, 11988 tbn, 23.98 tbc (default)
    Metadata:
      creation_time   : 2018-06-26T21:22:27.000000Z
      handler_name    : Apple Alias Data Handler
      encoder         : Lavc58.20.103 dnxhd
frame=   24 fps= 16 q=1.0 Lsize=   85345kB time=00:00:00.95 bitrate=728755.0kbits/s speed=0.647x
video:85344kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.001724%

[electron.rotoscope]

I should note the confusion is probably related to the DNxHD Codec 175X/185X/220X, which also has a name ending in X but should always be 10-bit, as can be seen on page 9 of ​http://www.avid.com/static/resources/US/documents/DNxHD.pdf

[Elon Musk]

This is just not correct.

[electron.rotoscope]

Replying to richardpl:

This is just not correct.

As in dnxhdenc.c is coded incorrectly? Because I agree! I honestly don't see this as an enhancement. As the current version is, it will only create out-of-spec files when using DNxHR HQX profile

[electron.rotoscope]

durandal_1707 provided a screenshot of a spec for the format (attached)

Looks like that spec allows both 10 bit and 12 bit, so ffmpeg should allow both

[Carl Eugen Hoyos]

FFmpeg correctly reports that libavcodec does not support encoding 12bit dnxhd, so this is a wish for an enhancement.

[Francesco Bucciantini]

Has there been any improvement in libavcodec over the last 4 years regarding 12bit encoding support for DNxHQX? If not, is there any plan to implement this?