JPEG 2000 images are decoded twice and needlessly slow

[Tomas Härdin]

If ffmpeg is told to decode a single JPEG 2000 image then it will decode it twice. If -lowres 2 is specified then it will first decode it at full resolution and then at the requested quarter resolution. For lossless JPEG 2000 this is especially slow.

$ time ./ffmpeg -loglevel error -lowres 2 -i smoke_sauna_JPEG2000_SDR_3840x2160p50_YUV444_12bit_Lossless.mov -vframes 1 -f framecrc -
#software: Lavf59.32.101
#tb 0: 1/50
#media_type 0: video
#codec_id 0: rawvideo
#dimensions 0: 960x540
#sar 0: 1/1
0,          0,          0,        1,  3110400, 0x5091f9f3

real    0m7,797s
user    0m9,841s
sys     0m0,441s

Dummying out the call to jpeg2000_decode_tile() for the first call to jpeg2000_decode_frame() still gives the same results, but much faster:

#software: Lavf59.32.101
#tb 0: 1/50
#media_type 0: video
#codec_id 0: rawvideo
#dimensions 0: 960x540
#sar 0: 1/1
0,          0,          0,        1,  3110400, 0x5091f9f3

real    0m1,300s
user    0m3,539s
sys     0m0,415s

All that the first decode really does is set avctx->pix_fmt. What I'd like to see:

• that jpeg2000_decode_frame() is only called once
• that jpeg2000_decode_frame() is called with the correct value for lowres
• that jpeg2000_decode_frame() is allowed to use threads during probing

The first could be accomplished by caching the decoded frame, or with a flag of some kind.

Commit: 686096739b129c7e3ea26be29c875e0887e58c49 since current master is broken in libavutil/x86/tx_float.asm
Sample: ​ftp://svtopencontent.svt.se/pub/svt_videotestsuite_natural_complexity/REC709/smoke_sauna_JPEG2000_SDR_3840x2160p50_YUV444_12bit_Lossless.mov

[mkver]

As I have already told you on IRC: The jpeg2000 decoder needs to set the FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM cap internal (and needs to export the relevant info even if the frame is discarded due to skip_frame being to set to AVDISCARD_ALL). Then the frame won't be decoded in avformat_find_stream_info().

[Tomas Härdin]

Replying to mkver:

As I have already told you on IRC: The jpeg2000 decoder needs to set the FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM cap internal (and needs to export the relevant info even if the frame is discarded due to skip_frame being to set to AVDISCARD_ALL). Then the frame won't be decoded in avformat_find_stream_info().

I implemented a very hackish solution like this and it does indeed work. Basically jpeg2000_read_main_headers(), ff_thread_get_buffer(), cleanup and then returning. I'm using pngdec.c as my reference.

[mkver]

Fixed in 793282adc2afe962a1bd97828468e499f247be3a.