change of matrix cannot happen on full range

[Balling]

Summary of the bug:

I have BT.709 sample that I want to do a jpeg image from, yet because std-complient jpeg is only bt.601 full range and thus the change of matrix must occur it just sees hevc video as bt.601 full range and does a no-op! Does not happen in png (remove gAMA from png). So apparently c523724c693ee031d8b5351aff6232a1f7d86876 was not enough!
How to reproduce:

PS C:\Users\xxxx>ffmpeg.exe  -i "this one does it.mp4" -frames:v 1 -q:v 1 318281.jpg
ffmpeg version N-106352-g2a1eae2179-20220319 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 11.2.0 (crosstool-NG 1.24.0.533_681aaef)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-version3 --disable-debug --enable-shared --disable-static --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --disable-avisynth --enable-libdav1d --disable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --disable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --disable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --disable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --disable-libx264 --disable-libx265 --disable-libxavs2 --disable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20220319
  libavutil      57. 24.101 / 57. 24.101
  libavcodec     59. 25.100 / 59. 25.100
  libavformat    59. 20.101 / 59. 20.101
  libavdevice    59.  6.100 / 59.  6.100
  libavfilter     8. 29.100 /  8. 29.100
  libswscale      6.  6.100 /  6.  6.100
  libswresample   4.  6.100 /  4.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'this one does it.mp4':
  Metadata:
    major_brand     : qt
    minor_version   : 0
    compatible_brands: qt
    creation_time   : 2021-07-05T19:03:03.000000Z
    com.apple.quicktime.description: [{"rParams":"2560000,{1080, 1920}","appRecord":"1","videoIndex":"1","userDevice":"iPhone11,8","userSystem":"14.2"}]
    com.apple.quicktime.software: {"TEEditor":"1"},{"isFastImport":"0"},{"transType":"2"},{"source":"1"}
    com.apple.quicktime.copyright: ba465c12a45eb7a886ad4f22aa7710b6
  Duration: 00:00:07.03, start: 0.000000, bitrate: 4004 kb/s
  Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 132 kb/s (default)
    Metadata:
      creation_time   : 2021-07-05T19:03:03.000000Z
      handler_name    : Core Media Audio
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, bt709), 720x1280, 3864 kb/s, 29.01 fps, 29 tbr, 600 tbn (default)
    Metadata:
      creation_time   : 2021-07-05T19:03:03.000000Z
      handler_name    : Core Media Video
      vendor_id       : [0][0][0][0]
      encoder         : HEVC
Stream mapping:
  Stream #0:1 -> #0:0 (hevc (native) -> mjpeg (native))
Press [q] to stop, [?] for help
Output #0, image2, to '318281.jpg':
  Metadata:
    major_brand     : qt
    minor_version   : 0
    compatible_brands: qt
    com.apple.quicktime.copyright: ba465c12a45eb7a886ad4f22aa7710b6
    com.apple.quicktime.description: [{"rParams":"2560000,{1080, 1920}","appRecord":"1","videoIndex":"1","userDevice":"iPhone11,8","userSystem":"14.2"}]
    com.apple.quicktime.software: {"TEEditor":"1"},{"isFastImport":"0"},{"transType":"2"},{"source":"1"}
    encoder         : Lavf59.20.101
  Stream #0:0(und): Video: mjpeg, yuvj420p(pc, bt709, progressive), 720x1280, q=2-31, 200 kb/s, 29 fps, 29 tbn (default)
    Metadata:
      creation_time   : 2021-07-05T19:03:03.000000Z
      handler_name    : Core Media Video
      vendor_id       : [0][0][0][0]
      encoder         : Lavc59.25.100 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
frame=    1 fps=0.0 q=1.0 Lsize=N/A time=00:00:00.03 bitrate=N/A speed=0.229x
video:105kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

The sample is here: ​​https://disk.yandex.ru/i/heY3zwPpDbv-cg

The downupstream issue is here ​https://github.com/n00mkrad/flowframes/issues/84
zscale is affected too, forcing -vf scale does not work.

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

[Balling]

As for png you need to remove gAMA chunk with pngcrush:

ffmpeg.exe -i "this one does it.mp4" -frames:v 1 -q:v 1 318281.png
pngcrush -m 40 -rem gAMA 318281.png withoutgAMA.png

I will attach reference image so as to help you guys.

[pdr0]

The downupstream issue is here ​​https://github.com/n00mkrad/flowframes/issues/84
zscale is affected too, forcing -vf scale does not work.

specifying zscale matrixin,matrix works ok

ffmpeg -i "this one does it.mp4" -frames:v 1 -q:v 1 -vf zscale=matrixin=709:matrix=170m zscale_colormatrix.jpg

[Balling]

Yes, zscale is only affected when you omit output matrix:

ffmpeg -i "this one does it.mp4" -frames:v 1 -q:v 1 -vf zscale=matrixin=709 zscale_colormatrix.jpg

But that is still bad, because that means here matrixin does not work.

[Balling]

And BTW you need -vf zscale=min=709:m=170m:cin=left:c=center or just -vf zscale=min=709:m=170m:c=center or it will not be perfect.

[Balling]

I think you can fix this fast. I suppose you did not take matrix into account in c523724c693ee031d8b5351aff6232a1f7d86876.

[Balling]

forcing -vf scale does not work

Forcing does work now!! I did preserve the version I used when opening the bug, it was broken (04cc7a5548fa22241a2d6bb4f719fe8e0e0843f1). Why no one notified me?

Anyway, we still need to support automatic forcing of it for jpeg that only supports bt.601.

ffmpeg.exe -i "this one does it.mp4" -vf scale=out_color_matrix=smpte170m -frames:v 1 -q:v 1 3182821211.jpg

Changing component since it is now not in swscale.

[Carl Eugen Hoyos]

Is this not reproducible with an input file that can be attached?

[Balling]

​https://mega.nz/folder/5EMWDDrS#MlG_UA9CHvG633ldHvmnTQ

Good enough for me.

[Balling]

Forcing scale actually does not work. OOPSIE

[Balling]

I noticed the sample is missing, so gonna reupload. ​https://disk.yandex.ru/i/heY3zwPpDbv-cg

[Balling]

Still broken. Haasn!!

[Balling]

Replying to Balling:

Forcing scale actually does not work. OOPSIE

Now it works again, strange stuff (no need to force color range to pc, btw, but I did not check)... new option out_chroma_loc also needs to be set to center, I did not check what happens

ffmpeg.exe -i "this one does it.mp4" -vf scale=out_color_matrix=smpte170m -frames:v 1 -q:v 1 -f md5 -

But there is obviously a need to force range when just doing md5:

ffmpeg.exe -i "this one does it.mp4" -vf scale=out_color_matrix=smpte170m -frames:v 1 -q:v 1 3182821211.jpg

[Balling]

The chroma sample location should be fixed with this patch when it will get eventually applied.

​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20250723135626.1390296-12-ffmpeg@haasn.xyz/

[Balling]

It is actually obvious that the issue happens here: when you force the output matrix -vf scale=out_color_matrix=smpte170m it prints

YUV color matrix differs for YUV->YUV, using intermediate RGB to convert

while it does not without it, while it should.