JPEG2000 Decode Problem

[smallishzulu]

Hi All,

I have a JPEG2000 stream output from Nimbra. The TS stream is in here:
​https://drive.google.com/file/d/1CxtlCT4bbaJB82WBu_PckcooiP6v7I3C/view?usp=sharing

When I try to probe, decode or transcode the stream, the resolution and fps values are misdetected. The source is actually 1920x1080i@50 JPEG2000 with 180 Mbps Video.

I am using the latest ffmpeg version (5.1).

Here is the log and command line:

ffprobe j2k.ts

ffprobe version 5.1-0ubuntu1~20.04.sav0 Copyright (c) 2007-2022 the FFmpeg developers

built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
configuration: --prefix=/usr --extra-version='0ubuntu1~20.04.sav0' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-lcms2 --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libdav1d --enable-libjxl --enable-librist --enable-libvmaf --enable-libzimg --enable-crystalhd --enable-libmfx --enable-libsvtav1 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libplacebo --enable-libx264 --enable-shared
libavutil 57. 28.100 / 57. 28.100
libavcodec 59. 37.100 / 59. 37.100
libavformat 59. 27.100 / 59. 27.100
libavdevice 59. 7.100 / 59. 7.100
libavfilter 8. 44.100 / 8. 44.100
libswscale 6. 7.100 / 6. 7.100
libswresample 4. 7.100 / 4. 7.100
libpostproc 56. 6.100 / 56. 6.100

[mpegts @ 0x55d7d6649ac0] Stream #0: not enough frames to estimate rate; consider increasing probesize
[mpegts @ 0x55d7d6649ac0] stream 0 : no PTS found at end of file, duration not set
[mpegts @ 0x55d7d6649ac0] stream 1 : no PTS found at end of file, duration not set
[mpegts @ 0x55d7d6649ac0] stream 2 : no PTS found at end of file, duration not set
Input #0, mpegts, from 'j2k.ts':

Duration: 00:00:17.16, start: 52309.086933, bitrate: 183838 kb/s
Program 1
Stream #0:0[0x200]: Video: jpeg2000 ([33][0][0][0] / 0x0021), yuv422p10le, 1920x540, 90k tbr, 90k tbn
Stream #0:1[0x294]: Audio: s302m (BSSD / 0x44535342), 48000 Hz, stereo, s32 (20 bit), 2304 kb/s
Stream #0:2[0x295]: Audio: s302m (BSSD / 0x44535342), 48000 Hz, stereo, s32 (20 bit), 2304 kb/s
Stream #0:3[0x240]: Data: bin_data (VANC / 0x434E4156)

[smallishzulu]

The stream is based on this standard:
​https://www.videoservicesforum.org/download/technical_recommendations/VSF_TR-01_2018-06-05.pdf

The source of the problem can be related with a flag as below ticket because 1080 / 2 = 540. This can be interlaced / progressive issue:
​https://trac.ffmpeg.org/ticket/4141

[Balling]

We do not support this. #1102

[smallishzulu]

Ok, well noted Balling. I have changed the source to progressive, but this time jpeg2000 decoder is not working properly. It does not decode all the frames. When I transcode, I get only a few frames and freezed image:

ffmpeg -i j2k25p1080.ts -c:v mpeg2video -b:v 1000k -an -f mpegts -y out.ts
ffmpeg version 5.1-0ubuntu1~20.04.sav0 Copyright (c) 2000-2022 the FFmpeg developers

built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
configuration: --prefix=/usr --extra-version='0ubuntu1~20.04.sav0' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-lcms2 --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libdav1d --enable-libjxl --enable-librist --enable-libvmaf --enable-libzimg --enable-crystalhd --enable-libmfx --enable-libsvtav1 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libplacebo --enable-libx264 --enable-shared
libavutil 57. 28.100 / 57. 28.100
libavcodec 59. 37.100 / 59. 37.100
libavformat 59. 27.100 / 59. 27.100
libavdevice 59. 7.100 / 59. 7.100
libavfilter 8. 44.100 / 8. 44.100
libswscale 6. 7.100 / 6. 7.100
libswresample 4. 7.100 / 4. 7.100
libpostproc 56. 6.100 / 56. 6.100

[mpegts @ 0x557581285200] Stream #0: not enough frames to estimate rate; consider increasing probesize
[mpegts @ 0x557581285200] PES packet size mismatch
[mpegts @ 0x557581285200] Packet corrupt (stream = 1, dts = 10800000).
[mpegts @ 0x557581285200] PES packet size mismatch
[mpegts @ 0x557581285200] Packet corrupt (stream = 2, dts = 10800000).
Input #0, mpegts, from 'j2k25p1080.ts':

Duration: 00:00:11.00, start: 109.000000, bitrate: 183128 kb/s
Program 1
Stream #0:0[0x200]: Video: jpeg2000 ([33][0][0][0] / 0x0021), yuv422p10le, 1920x1080, 90k tbr, 90k tbn
Stream #0:1[0x294]: Audio: s302m (BSSD / 0x44535342), 48000 Hz, stereo, s32 (20 bit), 2304 kb/s
Stream #0:2[0x295]: Audio: s302m (BSSD / 0x44535342), 48000 Hz, stereo, s32 (20 bit), 2304 kb/s
Stream #0:3[0x240]: Data: bin_data (VANC / 0x434E4156)

Stream mapping:

Stream #0:0 -> #0:0 (jpeg2000 (native) -> mpeg2video (native))

Press [q] to stop, ? for help
Output #0, mpegts, to 'out.ts':

Metadata:

encoder : Lavf59.27.100

Stream #0:0: Video: mpeg2video (4:2:2), yuv422p(tv, progressive), 1920x1080, q=2-31, 1000 kb/s, 240 fps, 90k tbn

Metadata:

encoder : Lavc59.37.100 mpeg2video

Side data:

cpb: bitrate max/min/avg: 0/0/1000000 buffer size: 0 vbv_delay: N/A

frame= 5 fps=2.4 q=6.2 Lsize= 998kB time=00:00:06.87 bitrate=1188.3kbits/s speed=3.29x
video:974kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.475384%

Source video:
​https://drive.google.com/file/d/17PXLGM1_Gq4sMEqNa7HoMpHtW03xFLJs/view?usp=sharing

Output:
​https://drive.google.com/file/d/1Ghd8MzMGMfR2ghQZSMEaZiA982XpPjCx/view?usp=sharing

Even if I decode to JPG images per frame, I get the same frame picture for almost all JPGs.

[Carl Eugen Hoyos]

For future tickets: PLease do not provide ffprobe output for tickets unrelated to ffprobe.

It seems to me that our j2k parser does not support the J2K Elementary Stream Header.

[smallishzulu]

I can offer a bounty for this work, if anyone is interested in.

[Carl Eugen Hoyos]

See table S.1