Opened 5 months ago

Last modified 8 weeks ago

#9769 open defect

JPEG 2000 decoder: decoder should ignore any data that appears after End of Codestream marker 0xFFD9

Reported by: Pierre-Anthony Lemieux Owned by:
Priority: minor Component: avcodec
Version: git-master Keywords: j2k
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description (last modified by Pierre-Anthony Lemieux)

Summary of the bug:

JPEG 2000 decoder outputs SOC marker not present on a JPEG 2000 codestream file (j2c) that contains bad bytes after the End of Codestream marker 0xFFD9.

See attached j2k1.j2c​ file.

A J2C file can contain only one codestreams, and thus bytes after the End of Codestream marker 0xFFD9 should be ignored.

How to reproduce:

ffmpeg -i j2k1.j2c out.png
ffmpeg version N-106674-gdcbf10acdc Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)

Attachments (8)

j2k1.j2c (2.8 KB ) - added by Pierre-Anthony Lemieux 5 months ago.
qjcxqdaws.png (6.2 KB ) - added by Balling 5 months ago.
openjpeg-output.png (7.0 KB ) - added by Pierre-Anthony Lemieux 5 months ago.
ffmpeg-output.png (5.7 KB ) - added by Pierre-Anthony Lemieux 5 months ago.
kakadu.jxl (2.1 KB ) - added by Balling 5 months ago.
j2k1.lossless.j2c (1.8 KB ) - added by Pierre-Anthony Lemieux 5 months ago.
j2k1.8_0_5.j2c (2.0 KB ) - added by Pierre-Anthony Lemieux 5 months ago.
j2000lossless.jxl (696 bytes ) - added by Balling 5 months ago.

Download all attachments as: .zip

Change History (30)

by Pierre-Anthony Lemieux, 5 months ago

Attachment: j2k1.j2c added

comment:1 by Balling, 5 months ago

Well, actually it does decode in both cases: -c:v libopenjpeg works too. But the result is not bitperfect between -c:v jpeg2000 and -c:v libopenjpeg

in reply to:  1 comment:2 by Pierre-Anthony Lemieux, 5 months ago

Replying to Balling:

Well, actually it does decode in both cases: -c:v libopenjpeg works too. But the result is not bitperfect between -c:v jpeg2000 and -c:v libopenjpeg

Are you using the file attached to the ticket against HEAD of master?

in reply to:  1 ; comment:3 by Pierre-Anthony Lemieux, 5 months ago

Replying to Balling:

Well, actually it does decode in both cases: -c:v libopenjpeg works too. But the result is not bitperfect between -c:v jpeg2000 and -c:v libopenjpeg

Are you using the file attached to the ticket against HEAD of master?

comment:4 by Pierre-Anthony Lemieux, 5 months ago

Just to make sure, I am at commit f3b7ba21ba49b32b4476a8c7c5a9bcdad15e3943.

in reply to:  3 comment:5 by Balling, 5 months ago

Replying to Pierre-Anthony Lemieux:

Replying to Balling:

Well, actually it does decode in both cases: -c:v libopenjpeg works too. But the result is not bitperfect between -c:v jpeg2000 and -c:v libopenjpeg

Are you using the file attached to the ticket against HEAD of master?

Yes, obviously. What is the expected output? The fact that libopenjpeg also does not like it and it is reference software of ISO... two 16 bit images differ only little and only around the 0 number. It has -1, -1, -1 in 16 bit and -2, 2, 2.

Last edited 5 months ago by Balling (previous) (diff)

by Balling, 5 months ago

Attachment: qjcxqdaws.png added

comment:6 by Pierre-Anthony Lemieux, 5 months ago

I am able to decode j2k1.j2c with openjpeg (576f721) without error. See attached openjpeg-output.png, which looks like qjcxqdaws.png -- the results are not bit exact, but that is expected with lossy J2K.

To clarify, decoding with ffmpeg also results in an image (ffmpeg-output.png), but with the error message:

[jpeg2000 @ 0x55b536e435c0] SOC marker not present

by Pierre-Anthony Lemieux, 5 months ago

Attachment: openjpeg-output.png added

by Pierre-Anthony Lemieux, 5 months ago

Attachment: ffmpeg-output.png added

comment:7 by Balling, 5 months ago

the results are not bit exact, but that is expected with lossy J2K.

In fact it is obviosuly not expected, but that is a known bug in our decoder or openjpeg. Need to compare with kakadu (will attach, .\kdu_expand.exe -i j2k1.j2c -o xqawsqa.ppm, then compress with ffmpeg to lossless jxl -f md5 is 55cfac05df63f74cadb245e4ce1ab86d) after all it was created with Kakadu-v8.0.3. Also nvJPEG2000 of Nvidia written in CUDA... And the standard.

but with the error message:

That is technically a warning since it still allows decoding. How did you get j2k-samples/j2k1.j2c? What created it? Is it public test suite? Kakadu-v8.0.3 is not latest, 8.0.5 is. Here is kakadu: https://github.com/uclouvain/openjpeg/commit/92c0471154960d50ebe68b363f4bb1e2c1e4d775

by Balling, 5 months ago

Attachment: kakadu.jxl added

comment:8 by Balling, 5 months ago

Keywords: jpeg2000 removed
Reproduced by developer: set
Status: newopen

BTW, it is hillarious how lossless jpeg XL is smaller than lossy source jpeg 2000. HAHAHA. Love it! You can try it here. https://github.com/ValZapod/FFmpeg-Builds/releases/tag/latest

Of course Kakadu is again not lossless with the two others too. Sigh.

Last edited 5 months ago by Balling (previous) (diff)

comment:9 by Pierre-Anthony Lemieux, 5 months ago

TW, it is hillarious how lossless jpeg XL is smaller than lossy source jpeg 2000. HAHAHA. Love it!

I am not sure what is hilarious. The lossy J2K was created with a target bit rate, which the encoder try to hit.

The lossless J2K is smaller, and in fact smaller than JXL.

[edit: the `j2k1.j2c​` file is larger in part because it contains extraneous bad bytes after the end of the codestream]

Last edited 5 months ago by Pierre-Anthony Lemieux (previous) (diff)

by Pierre-Anthony Lemieux, 5 months ago

Attachment: j2k1.lossless.j2c added

comment:10 by Pierre-Anthony Lemieux, 5 months ago

the results are not bit exact, but that is expected with lossy J2K.

JPEG 2000 does not specify a bit-exact decoder (for lossy codestreams), which is why some deviation is expected across decoders and across decoding paths in a given decoder.

ISO/IEC 15444-4 | Rec. ITU-T T.803 (J2K conformance testing) specifies tolerances for decoding of lossy codestreams.

comment:11 by Pierre-Anthony Lemieux, 5 months ago

How did you get j2k-samples/j2k1.j2c? What created it?

It is a test file I've had for a while. According to the headers it looks like it was created by Kakadu 8.0.3. Using the latest demo executables, I get j2k1.8_0_5.j2c, which decodes to the same pixels as j2k1.j2c, but is smaller.

Looking at the differences, it looks like, for whatever reason, j2k1.j2c contains bad bytes beyond the first FFD9 (end of codestream marker).

So I suggest changing the title of the issue to "J2C decoder should ignore any data that appears after End of Codestream marker 0xFFD9".

by Pierre-Anthony Lemieux, 5 months ago

Attachment: j2k1.8_0_5.j2c added

comment:12 by Pierre-Anthony Lemieux, 5 months ago

Description: modified (diff)
Summary: JPEG 2000 decoder: SOC marker not presentJPEG 2000 decoder: decoder should ignore any data that appears after End of Codestream marker 0xFFD9

comment:13 by Balling, 5 months ago

The lossless J2K is smaller, and in fact smaller than JXL.

That is too be expected, you introduce some insane spatial and temporal artifacts when you do lossy encoding of JPEG2000 (see wikipedia https://en.wikipedia.org/wiki/JPEG_2000#/media/File:Jpeg2000_2-level_wavelet_transform-lichtenstein.png). Lossless encoding of those artifacts is a very hard thing. Now if you have lets says SVG vector source (be it OTF of that zero on the black background) (that is not yet rasterised) that is even less data. I will attach new jxl of your lossless j2c that is again even smaller (696 bytes vs 1 846 bytes).

Lossy j2k1.8_0_5.j2c does not have such a problem with SOF, so it is kinda obvious this is a bug with Kakadu.

for whatever reason, j2k1.j2c contains bad bytes beyond the first FFD9 (end of codestream marker).

Oh. Okay, but that is still a bug in Kakadu.

ISO/IEC 15444-4 | Rec. ITU-T T.803 (J2K conformance testing) specifies tolerances for decoding of lossy codestreams.

Same happens in normal JPEG library. There is a difference in decoding with old stable and new libraries. We in ffmpeg use the old stable algorithm, this off-by-one should be fixed by comparing to original.

Last edited 5 months ago by Balling (previous) (diff)

by Balling, 5 months ago

Attachment: j2000lossless.jxl added

comment:14 by Balling, 5 months ago

BTW, if we are assuming that lossless image you provided is the original, then we can say which of Kakadu, OPEN jpeg or jpeg2000 decoders are best. Now, if we will look into very first line in the image jpeg 2000 native decoder has 13 pixels that are different with lossless, Kakadu has 2 and open jpeg has 12. So...

In yuview you can preserve all 16*3 bits and select black for no difference and white for difference and then compare in beyond compare...

Last edited 5 months ago by Balling (previous) (diff)

comment:15 by Balling, 5 months ago

The warnings are because of garbage in the end! Indeed see ffprobe.exe -show_frames -v debug j2k1.j2c it warns on second frame or what it thinks is a second frame.

As to whether it should read past the 0xFFD9, after all cannot it have multiple images in one j2c?

I will also point out that two encoded with two Kakadu versions are the same, except for some stuff after 0xFFD9 where should be nothing anyway.

I also showed that the Kakadu is better, BTW.

Last edited 5 months ago by Balling (previous) (diff)

comment:16 by Pierre-Anthony Lemieux, 5 months ago

Description: modified (diff)

comment:17 by Pierre-Anthony Lemieux, 5 months ago

after all cannot it have multiple images in one j2c?

As far as I know, a J2C file can contain only one codestream.

Thanks for the help debug the issue. I have updated the description.

comment:18 by Balling, 5 months ago

As far as I know, a J2C file can contain only one codestream.

It is jpx that can! https://github.com/uclouvain/openjpeg/issues/464

I also encoded with latest openjpeg new lossy image from your lossless image and reported a bug against openjpeg. I mean Kakadu is still slightly better than jpeg2000 (5 pixels better, LOL) but jpeg2000 is much better than openjpeg decoder. See https://github.com/uclouvain/openjpeg/issues/1420

Last edited 5 months ago by Balling (previous) (diff)

in reply to:  description ; comment:20 by Carl Eugen Hoyos, 8 weeks ago

Priority: normalminor

Replying to Pierre-Anthony Lemieux:

A J2C file can contain only one codestreams, and thus bytes after the End of Codestream marker 0xFFD9 should be ignored.

Why do you think so? Can you point to the specification saying so?

in reply to:  20 comment:21 by Balling, 8 weeks ago

Replying to Carl Eugen Hoyos:

Replying to Pierre-Anthony Lemieux:

A J2C file can contain only one codestreams, and thus bytes after the End of Codestream marker 0xFFD9 should be ignored.

Why do you think so? Can you point to the specification saying so?

Look who is back! The point here is that is fixed is Kakadu 8.0.5, sample is otherwise bitperfect. 8.0.3 was inserting garbage after FFD9.

in reply to:  20 comment:22 by Pierre-Anthony Lemieux, 8 weeks ago

Replying to Carl Eugen Hoyos:

Replying to Pierre-Anthony Lemieux:

A J2C file can contain only one codestreams, and thus bytes after the End of Codestream marker 0xFFD9 should be ignored.

Why do you think so? Can you point to the specification saying so?

A JPEG 2000 codestream is defined as starting with the SOC marker and ending with an EOC marker ("EOC indicates the end of the codestream" in ITU t.800), so bytes following the EOC marker can be ignored.

Some applications might purposefully add bytes after the EOC, e.g., in the case of constant bitrate streaming where the same amount of bytes (over the wire) is used for all codestreams.

Note: See TracTickets for help on using tickets.