scale ignores AVFrame's input range

[Diederick Niehorster]

Discovered when investigating #9573.

When reading a file with yuv420p10 grayscale data in full range (color_range=2) and requesting conversion to gray10, conversion does not use full range. You need to tell the scale filter to assume the input is full range with in_range=pc, despite the AVFrames being delivered to the scale_frame() function of the filter having color_range=2 (full/pc/JPEG range set).

Transcode command:

ffmpeg -y -i test3.mp4 -vf format=gray10 -c:v ffv1 -level 3 test2.mkv

If using

ffmpeg -y -i test3.mp4 -vf scale=in_range=pc,format=gray10 -c:v ffv1 -level 3 test2.mkv

everything is fine.

Command to see if video encoded fine:

ffmpeg -i test3.mp4 -i test2.mkv -lavfi  "[0:v]extractplanes=y,settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr" -f null -

which yields PSNR y:26.424073 for the first command, and PSNR y:inf for the second.

I don't quite understand why in_range=pc is needed. when looking at line 802 of vf_scale.c, it does detect that the input is full range (i stepped through the code, it does, in_full is set to 1), and this seems to be set on the scale context in lines 809-816, but apparently it doesn't do anything.

I'm not sure if this is a defect or an enhancement request, but i think since all the info is there and since sws_setColorspaceDetails() on lines 809-816 doesn't seem to do what it should (set input range of the scaler to full), this is a defect.

NB: out_range doesn't need to be set as its set to full range automatically for gray10 input due to handle_jpeg() in utils.c of libswscale

full ffmpeg output of first command

ffmpeg -y -i test3.mp4 -vf format=gray10 -c:v ffv1 -level 3 test2.mkv
ffmpeg version 2021-12-27-git-617452ce2c-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11.2.0 (Rev2, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      57. 13.100 / 57. 13.100
  libavcodec     59. 15.101 / 59. 15.101
  libavformat    59. 10.100 / 59. 10.100
  libavdevice    59.  1.100 / 59.  1.100
  libavfilter     8. 21.100 /  8. 21.100
  libswscale      6.  1.102 /  6.  1.102
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test3.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf59.10.100
  Duration: 00:00:00.02, start: 0.000000, bitrate: 1054055 kb/s
  Stream #0:0[0x1](und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv420p10le(pc, progressive), 1152x390, 1053650 kb/s, 500 fps, 500 tbr, 16k tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> ffv1 (native))
Press [q] to stop, [?] for help
[ffv1 @ 000002e1c5d80340] bits_per_raw_sample > 8, forcing range coder
Output #0, matroska, to 'test2.mkv':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf59.10.100
  Stream #0:0(und): Video: ffv1 (FFV1 / 0x31564646), gray10le(pc, progressive), 1152x390, q=2-31, 200 kb/s, 500 fps, 1k tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc59.15.101 ffv1
frame=    9 fps=0.0 q=-0.0 Lsize=    2175kB time=00:00:00.01 bitrate=1048205.6kbits/s speed=0.151x
video:2174kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.050172%

[Diederick Niehorster]

input file

[Balling]

Some comments. First of all this fixes it, so that part is just a duplicate of #8862:

ffplay test3.mp4 -vf scale=in_color_matrix=auto,format=gbrp
Yes, first command thinks input is limited and writes full range data thus 255, 255, 255 is preserved, but everything else is bad.

[Balling]

Does the patch ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220303120645.13076-1-martin@martin.st/ also fixed ffplay -i? Compare to correct ffplay test3.mp4 -vf scale=in_color_matrix=auto,format=gbrp

[Balling]

Other samples affected in the same way as this and BTW what a joke, they need ffplay -vf scale=in_color_matrix=auto,format=gbrp

​https://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket524/AVCI50.mov

​https://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket524/AVCI100.mov

They both have Y component of 249 (first file) and 243 (second), so it may as well be full range. First file has some black at 13, BTW, too.

I cannot say 100% whether it is really full and not limited range.

[Diederick Niehorster]

fixed in e645a1dd

[Diederick Niehorster]

fixed in e645a1dd

[Balling]

ffplay is also fixed now after e645a1ddb90a863e129108aad9aa7e2d417f3615, at least in this topic (AVCI50.mov + AVCI100.mov). test3.mp4 is also fixed in ffplay, very nice! Will check what part of #8862 is fixed.