Overflows (?) in xyz -> rgb conversion

[Carl Eugen Hoyos]

​​http://thread.gmane.org/gmane.comp.video.ffmpeg.user/58666
The sample attached to ticket #4827 is in fact decoded fine (at least visually) by the native jpeg 2000 decoder but the colourspace conversion shows posterization artefacts.
This is also reproducible with the libopenjpeg decoder.

$ ffmpeg -i frm_000127.j2c out.png
ffmpeg version N-74866-g0cdba4a Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
[jpeg2000 @ 0x33bd940] End mismatch 1
    Last message repeated 4 times
[jpeg2000 @ 0x33bd940] End mismatch 2
    Last message repeated 1 times
[jpeg2000 @ 0x33bd940] End mismatch 1
    Last message repeated 28 times
[jpeg2000 @ 0x33bd940] End mismatch 2
[jpeg2000 @ 0x33bd940] End mismatch 1
    Last message repeated 19 times
Input #0, j2k_pipe, from 'frm_000127.j2c':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: jpeg2000 (JPEG 2000 digital cinema 2K), xyz12le, 2048x858, 25 tbr, 25 tbn, 25 tbc
Output #0, image2, to 'out.png':
  Metadata:
    encoder         : Lavf56.40.101
    Stream #0:0: Video: png, rgb48be, 2048x858, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc56.60.100 png
Stream mapping:
  Stream #0:0 -> #0:0 (jpeg2000 (native) -> png (native))
Press [q] to stop, [?] for help
[jpeg2000 @ 0x3655fa0] End mismatch 1
    Last message repeated 4 times
[jpeg2000 @ 0x3655fa0] End mismatch 2
    Last message repeated 1 times
[jpeg2000 @ 0x3655fa0] End mismatch 1
    Last message repeated 28 times
[jpeg2000 @ 0x3655fa0] End mismatch 2
[jpeg2000 @ 0x3655fa0] End mismatch 1
    Last message repeated 19 times
frame=    1 fps=0.8 q=-0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
video:5354kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

$ ffmpeg -vcodec libopenjpeg -pix_fmt xyz12le -i frm_000127.j2c out.png
ffmpeg version N-74866-g0cdba4a Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --enable-libopenjpeg
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Input #0, j2k_pipe, from 'frm_000127.j2c':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: jpeg2000, xyz12le, 2048x858, 25 tbr, 25 tbn, 25 tbc
Output #0, image2, to 'out.png':
  Metadata:
    encoder         : Lavf56.40.101
    Stream #0:0: Video: png, rgb48be, 2048x858, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc56.60.100 png
Stream mapping:
  Stream #0:0 -> #0:0 (jpeg2000 (libopenjpeg) -> png (native))
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
video:5354kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

I don't know if this ticket is related to ticket #3643.

[Elon Musk]

The sample attached in ticket looks fine to me, do you have some other samples?

[Carl Eugen Hoyos]

The command line here still produces a png with massive artefacts (posterization) in the top left corner with current FFmpeg git head, the artefacts do not appear without a colour space conversion, see ticket #4827.
This may or may not be a duplicate of ticket #3643.

[Balling]

in fact decoded fine (at least visually)

Oh really? The only way to preserve the xyz12le stuff is rawvideo (see last comment in #1279) and it is not lossless compared to -c:v libopenjpeg. I.e. file produced by (-pixel_format xyz12le is needed for libopenjpeg before ​https://github.com/uclouvain/openjpeg/pull/1200 is applied):

ffmpeg -pixel_format xyz12le -c:v libopenjpeg -i frm_000127.j2c -f rawvideo out

is not the same as:

ffmpeg -i frm_000127.j2c -f rawvideo out

has off-by-one error somewhere that breaks every 32 byte if you compare in hex diff. It is kind of hard for me to say what decoder is correct.

a png with massive artefacts (posterization) in the top left

So what? Of course, the artifacts can be in original xyz12le file.

[Balling]

So what? Of course, the artifacts can be in original xyz12le file.

And it looks like it is possible. Here in leaked "My little pony" in DNxHD 175x (magnet magnet:?xt=urn:btih:215babe2205786dbc77dd58607fc4490302112c3 /Movie folder) you have the same Posterization in the very same scene even though the file is 10 bit SDR with very high quality.

[Elon Musk]

Please do not post magnet links here in any form whatsoever. Thanks.

[Balling]

has off-by-one error somewhere that breaks every 32 byte if you compare in hex diff

I learnt that lossy jpeg 2000 is not required to be decoded bitperfectly by the standard itself with different jpeg 2000 decoders, in fact that is the case with kakadu vs openjpeg vs our jpeg2000. Sigh. Faceplam. ​https://github.com/uclouvain/openjpeg/issues/1420

[Balling]

There is now some change here: 9d8f5141cf7da4c72e8e1a08e9a967c0f163a78a

[Niklas Haas]

Fix: https://ffmpeg.org//pipermail/ffmpeg-devel/2024-December/337540.html

[Balling]

Replying to Balling:

has off-by-one error somewhere that breaks every 32 byte if you compare in hex diff

I learnt that lossy jpeg 2000 is not required to be decoded bitperfectly by the standard itself with different jpeg 2000 decoders, in fact that is the case with kakadu vs openjpeg vs our jpeg2000. Sigh. Faceplam. ​https://github.com/uclouvain/openjpeg/issues/1420

That is no longer true. Since genius patch -flags +bitexact can be used to actually output true lossy decoding. 82467b635efced67c1767cb810af1f3c31a2e493

[Balling]

Re mailing list:

There is a chance we will move to XYZ media soon, since it is the only level left after PQ 12 bit 8K video. So better to hardcode it, as it will be faster and optimised by compilers better, maybe even -march=native and x86_64-v3 will work cute, see this: ​https://github.com/zhongfly/mpv-winbuild/releases/tag/2024-12-21-d827019 (ffmpeg-x86_64-v3-git-095f8038f.7z, nice)

[Niklas Haas <git@haasn.dev>]

In af6d52ee/ffmpeg:

swscale: use 16-bit intermediate precision for RGB/XYZ conversion

The current logic uses 12-bit linear light math, which is woefully insufficient
and leads to nasty postarization artifacts. This patch simply switches the
internal logic to 16-bit precision.

This raises the memory requirement of these tables from 32 kB to 272 kB.

All relevant FATE tests updated for improved accuracy.

Fixes: #4829
Signed-off-by: Niklas Haas <​git@haasn.dev>
Sponsored-by: Sovereign Tech Fund