FFmpeg’s nv12 libx264: ffmpeg encodes in “yuv420p” instead of “nv12”

[Kdmeizk]

Input file:
​https://drive.google.com/file/d/0B40VEgqICZzWZ1UwQ2M5VldvNmM/view?usp=sharing

ffmpeg-20191101-53c21c2-win64-static\bin\ffmpeg.exe -i 00006.m2ts -to 00:00:05 -c:v libx264 -pix_fmt nv12 -an output.mkv
ffmpeg version git-2019-11-01-53c21c2 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 9.2.1 (GCC) 20191010
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
  libavutil      56. 35.101 / 56. 35.101
  libavcodec     58. 60.100 / 58. 60.100
  libavformat    58. 33.100 / 58. 33.100
  libavdevice    58.  9.100 / 58.  9.100
  libavfilter     7. 66.100 /  7. 66.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, mpegts, from '00006.m2ts':
  Duration: 00:00:44.05, start: 1.992667, bitrate: 36799 kb/s
  Program 1 
    Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 90k tbn, 47.95 tbc
    Stream #0:1[0x1100]: Audio: dts (DTS-HD MA) ([134][0][0][0] / 0x0086), 48000 Hz, 5.1(side), s32p (24 bit)
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
Output #0, matroska, to 'output.mkv':
  Metadata:
    encoder         : Lavf58.33.100
    Stream #0:0: Video: h264 (libx264) (H264 / 0x34363248), nv12, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 23.98 fps, 1k tbn, 23.98 tbc
    Metadata:
      encoder         : Lavc58.60.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
video:1408kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.115162%

ffmpeg-20191101-53c21c2-win64-static\bin\ffprobe.exe output.mkv
ffprobe version git-2019-11-01-53c21c2 Copyright (c) 2007-2019 the FFmpeg developers
  built with gcc 9.2.1 (GCC) 20191010
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
  libavutil      56. 35.101 / 56. 35.101
  libavcodec     58. 60.100 / 58. 60.100
  libavformat    58. 33.100 / 58. 33.100
  libavdevice    58.  9.100 / 58.  9.100
  libavfilter     7. 66.100 /  7. 66.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, matroska,webm, from 'output.mkv':
  Metadata:
    ENCODER         : Lavf58.33.100
  Duration: 00:00:05.01, start: 0.000000, bitrate: 2306 kb/s
    Stream #0:0: Video: h264 (High), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
    Metadata:
      ENCODER         : Lavc58.60.100 libx264
      DURATION        : 00:00:05.005000000

[Hendrik]

The h264 bitstream carries chroma always in the same way. How a decoder chooses to represent it is up to the implementation, and the ffmpeg h264 decoder choose yuv420p to do it.

[Kdmeizk]

Replying to heleppkes:

The h264 bitstream carries chroma always in the same way. How a decoder chooses to represent it is up to the implementation, and the ffmpeg h264 decoder choose yuv420p to do it.

The following output files are byte-to-byte identical:

ffmpeg-20191101-53c21c2-win64-static\bin\ffmpeg.exe -i 00006.m2ts -to 00:00:05 -c:v libx264 -pix_fmt yuv420p -crf 0 -an output_yuv420p.nut
ffmpeg-20191101-53c21c2-win64-static\bin\ffmpeg.exe -i 00006.m2ts -to 00:00:05 -c:v libx264 -pix_fmt nv12 -crf 0 -an output_nv12.nut

But it is not normal.
From “​https://wiki.videolan.org/YUV/”:

For a single I420 pixel: YYYYYYYY UU VV
[…]
For 1 NV12 pixel: YYYYYYYY UVUV

So ffmpeg encodes in “yuv420p” instead of “nv12”, otherwise the files should not be byte-to-byte identical.

[Hendrik]

h264 does not care what format you feed it, it simply encodes 4:2:0 chroma. It does not encode a specific pixel format, those are implementation details.

[Kdmeizk]

Replying to heleppkes:

h264 does not care what format you feed it, it simply encodes 4:2:0 chroma. It does not encode a specific pixel format, those are implementation details.

Example:

x264-r2969-d4099dd.exe --fullhelp
[…]
      --input-csp <string>    Specify input colorspace format for raw input
                              - valid csps for `raw' demuxer:
                                i400, i420, yv12, nv12, nv21, i422, yv16, nv16, 
                                yuyv, uyvy, i444, yv24, bgr, bgra, rgb
                              - valid csps for `lavf' demuxer:
                                yuv420p, yuyv422, rgb24, bgr24, yuv422p, 
                                yuv444p, yuv410p, yuv411p, gray, monow, monob, 
                                pal8, yuvj420p, yuvj422p, yuvj444p, uyvy422, 
                                uyyvyy411, bgr8, bgr4, bgr4_byte, rgb8, rgb4, 
                                rgb4_byte, nv12, nv21, argb, rgba, abgr, bgra, 
                                gray16be, gray16le, yuv440p, yuvj440p, 
                                yuva420p, rgb48be, rgb48le, rgb565be, rgb565le, 
                                rgb555be, rgb555le, bgr565be, bgr565le, 
                                bgr555be, bgr555le, vaapi_moco, vaapi_idct, 
                                vaapi_vld, yuv420p16le, yuv420p16be, 
                                yuv422p16le, yuv422p16be, yuv444p16le, 
                                yuv444p16be, dxva2_vld, rgb444le, rgb444be, 
                                bgr444le, bgr444be, ya8, bgr48be, bgr48le, 
                                yuv420p9be, yuv420p9le, yuv420p10be, 
                                yuv420p10le, yuv422p10be, yuv422p10le, 
                                yuv444p9be, yuv444p9le, yuv444p10be, 
                                yuv444p10le, yuv422p9be, yuv422p9le, gbrp, 
                                gbrp9be, gbrp9le, gbrp10be, gbrp10le, gbrp16be, 
                                gbrp16le, yuva422p, yuva444p, yuva420p9be, 
                                yuva420p9le, yuva422p9be, yuva422p9le, 
                                yuva444p9be, yuva444p9le, yuva420p10be, 
                                yuva420p10le, yuva422p10be, yuva422p10le, 
                                yuva444p10be, yuva444p10le, yuva420p16be, 
                                yuva420p16le, yuva422p16be, yuva422p16le, 
                                yuva444p16be, yuva444p16le, vdpau, xyz12le, 
                                xyz12be, nv16, nv20le, nv20be, rgba64be, 
                                rgba64le, bgra64be, bgra64le, yvyu422, ya16be, 
                                ya16le, gbrap, gbrap16be, gbrap16le, qsv, mmal, 
                                d3d11va_vld, cuda, 0rgb, rgb0, 0bgr, bgr0, 
                                yuv420p12be, yuv420p12le, yuv420p14be, 
                                yuv420p14le, yuv422p12be, yuv422p12le, 
                                yuv422p14be, yuv422p14le, yuv444p12be, 
                                yuv444p12le, yuv444p14be, yuv444p14le, 
                                gbrp12be, gbrp12le, gbrp14be, gbrp14le, 
                                yuvj411p, bayer_bggr8, bayer_rggb8, 
                                bayer_gbrg8, bayer_grbg8, bayer_bggr16le, 
                                bayer_bggr16be, bayer_rggb16le, bayer_rggb16be, 
                                bayer_gbrg16le, bayer_gbrg16be, bayer_grbg16le, 
                                bayer_grbg16be, xvmc, yuv440p10le, yuv440p10be, 
                                yuv440p12le, yuv440p12be, ayuv64le, ayuv64be, 
                                videotoolbox_vld, p010le, p010be, gbrap12be, 
                                gbrap12le, gbrap10be, gbrap10le, mediacodec, 
                                gray12be, gray12le, gray10be, gray10le, p016le, 
                                p016be, d3d11, gray9be, gray9le, gbrpf32be, 
                                gbrpf32le, gbrapf32be, gbrapf32le, drm_prime, 
                                opencl, gray14be, gray14le, grayf32be, 
                                grayf32le, yuva422p12be, yuva422p12le, 
                                yuva444p12be, yuva444p12le
      --output-csp <string>   Specify output colorspace ["i420"]
                                  - i400, i420, i422, i444, rgb

“-pix_fmt yuv420p” means “--input-csp yuv420p” and “--output-csp i420”.

“i420”	= colorspace format
“yuv420p”	= pixel format using the colorspace format “i420”

[Balling]

There are some ways to affect the chroma sitting/location/lattice in libx264. Keywords for googling chromaloc=2 chromaloc-top and chroma_sample_location. But you do NOT really need this. This is very obscure at least for AVC, for HEVC two chroma lattices are actively used. There are more than that though.

Lattice is signalled in VUI.

Also see ​https://forum.doom9.org/showpost.php?p=1766645&postcount=123

-vf 'scale=out_color_matrix=bt709:out_h_chr_pos=0:out_v_chr_pos=128'
for center left that is used in SDR. In HDR out_h_chr_pos=0:out_v_chr_pos=0 is in use. That is top left.

There are 4 more types of it. In jpeg, for example.

[Balling]

Oh, BTW, you can decode to rawvideo in format nv12 (HW decoders do that by default) and then encode NV12 but tagging it as yuv420p. The result will be very funny, but you can do that.