Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#3745 closed defect (invalid)

error with libopenjpeg encoding

Reported by: dericed Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: libopenjpeg
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

When using ffmpeg with libopenjpeg to make lossless encodings I use framemd5 to verify the losslessness. I found this sample that causes a glitch.

How to reproduce:

This command makes both a jpeg2000 output and a framemd5 output of the source file, which is available here https://archive.org/download/v210_cant_be_jpeg2000d/v210_cant_be_jpeg2000d.mov.

ffmpeg -y -i v210_cant_be_jpeg2000d.mov -c:v libopenjpeg -an isthisjpeg2000lossless.mov -f framemd5 -an v210_cant_be_jpeg2000d.framemd5
ffmpeg version N-64075-g47c8d1e Copyright (c) 2000-2014 the FFmpeg developers
  built on Jun 19 2014 10:18:25 with Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-nonfree --enable-hardcoded-tables --enable-avresample --enable-vda --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libfaac --enable-libmp3lame --enable-libxvid --enable-libfreetype --enable-ffplay --enable-libopenjpeg --extra-cflags='-I/usr/local/Cellar/openjpeg/1.5.1_1/include/openjpeg-1.5 '
  libavutil      52. 89.100 / 52. 89.100
  libavcodec     55. 67.100 / 55. 67.100
  libavformat    55. 44.100 / 55. 44.100
  libavdevice    55. 13.101 / 55. 13.101
  libavfilter     4.  8.100 /  4.  8.100
  libavresample   1.  3.  0 /  1.  3.  0
  libswscale      2.  6.100 /  2.  6.100
  libswresample   0. 19.100 /  0. 19.100
  libpostproc    52.  3.100 / 52.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'v210_cant_be_jpeg2000d.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf55.44.100
  Duration: 00:00:01.02, start: 0.021333, bitrate: 219267 kb/s
    Stream #0:0(eng): Video: v210 (v210 / 0x30313276), yuv422p10le, 720x486, 223725 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 30k tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc55.68.100 v210
    Stream #0:1(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 123 kb/s (default)
    Metadata:
      handler_name    : DataHandler
[mov @ 0x7fb81b00e400] Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead.
[framemd5 @ 0x7fb81b01f200] Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead.
Output #0, mov, to 'isthisjpeg2000lossless.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf55.44.100
    Stream #0:0(eng): Video: jpeg2000 (libopenjpeg) (mjp2 / 0x32706A6D), yuv422p10le, 720x486, q=2-31, 200 kb/s, 29.97 fps, 30k tbn, 29.97 tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc55.67.100 libopenjpeg
Output #1, framemd5, to 'v210_cant_be_jpeg2000d.framemd5':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf55.44.100
    Stream #1:0(eng): Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le, 720x486, q=2-31, 200 kb/s, 29.97 fps, 29.97 tbn, 29.97 tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc55.67.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (v210 (native) -> jpeg2000 (libopenjpeg))
  Stream #0:0 -> #1:0 (v210 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
frame=   31 fps=8.8 q=0.0 Lq=0.0 size=   13587kB time=00:00:01.03 bitrate=107608.2kbits/s dup=1 drop=0    
video:54593kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

To verify the losslessness I then make a framemd5 from the j2k output:

ffmpeg -y -vcodec libopenjpeg -i isthisjpeg2000lossless.mov -f framemd5 -an isthisjpeg2000lossless.framemd5
ffmpeg version N-64075-g47c8d1e Copyright (c) 2000-2014 the FFmpeg developers
  built on Jun 19 2014 10:18:25 with Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-nonfree --enable-hardcoded-tables --enable-avresample --enable-vda --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libfaac --enable-libmp3lame --enable-libxvid --enable-libfreetype --enable-ffplay --enable-libopenjpeg --extra-cflags='-I/usr/local/Cellar/openjpeg/1.5.1_1/include/openjpeg-1.5 '
  libavutil      52. 89.100 / 52. 89.100
  libavcodec     55. 67.100 / 55. 67.100
  libavformat    55. 44.100 / 55. 44.100
  libavdevice    55. 13.101 / 55. 13.101
  libavfilter     4.  8.100 /  4.  8.100
  libavresample   1.  3.  0 /  1.  3.  0
  libswscale      2.  6.100 /  2.  6.100
  libswresample   0. 19.100 /  0. 19.100
  libpostproc    52.  3.100 / 52.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'isthisjpeg2000lossless.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf55.44.100
  Duration: 00:00:01.04, start: 0.000000, bitrate: 107542 kb/s
    Stream #0:0(eng): Video: jpeg2000 (mjp2 / 0x32706A6D), yuv422p10le, 720x486, 107601 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 30k tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc55.67.100 libopenjpeg
[framemd5 @ 0x7f859d811e00] Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead.
Output #0, framemd5, to 'isthisjpeg2000lossless.framemd5':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf55.44.100
    Stream #0:0(eng): Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le, 720x486, q=2-31, 200 kb/s, 29.97 fps, 29.97 tbn, 29.97 tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc55.67.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (jpeg2000 (libopenjpeg) -> rawvideo (native))
Press [q] to stop, [?] for help
frame=   31 fps= 15 q=0.0 Lsize=       3kB time=00:00:01.03 bitrate=  20.3kbits/s    
video:42373kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

Then check the similarity of the framemd5 outputs:

diff v210_cant_be_jpeg2000d.framemd5 isthisjpeg2000lossless.framemd5 
6a7
> 0,          0,          0,        1,  1399680, b49bc926bc8d4734e861297e9bf63c96
23c24
< 0,         17,         17,        1,  1399680, 698ea483de551f4efc0cc9fbdebdcf61
---
> 0,         17,         17,        1,  1399680, 160cda3daf1402700944d2eaa322687e

With mov I often see this extra frame at the beginning, but I'm more concerned about frame 17. When I look at frame 17 in the j2k output it has bright red and green colors over the frame. The Y and U planes seem accurate but the V plane seems very odd. I posted an image of it here: https://twitter.com/dericed/status/482195897456275457 the image on the right shows U on the left and V on the right. Very odd. I reran the process hoping it was a fluke but got a consistently glitchy output for jpeg2000, though when I transcode the same source to ffv1 the framemd5s matched.

Attachments (2)

out17.j2k (436.7 KB) - added by cehoyos 4 years ago.
out17.tiff (2.0 MB) - added by dericed 4 years ago.

Change History (11)

comment:1 Changed 4 years ago by cehoyos

If I understand correctly, you see two different (and indpendent) issues:
One is that the FFmpeg mov muxer only outputs constant frame rate, so if FFmpeg interprets your input file as variable frame rate, it will insert frames to achieve constant frame rate. This is reported in several tickets.
-vsync 0 avoids frame duplications, it may brake A/V sync but to test the bitexactness of your encodings, it is useful.

I am unable to reproduce a difference between the original frames and the frames encoded with openjpeg-1.5.0-2.7.1.x86_64:

$ ffmpeg -vsync 0 -i v210_cant_be_jpeg2000d.mov -an -f crc -
ffmpeg version N-64232-g932be89 Copyright (c) 2000-2014 the FFmpeg developers
  built on Jun 26 2014 20:31:10 with gcc 4.7 (SUSE Linux)
  configuration: --enable-libopenjpeg --disable-decoder=jpeg2000
  libavutil      52. 90.101 / 52. 90.101
  libavcodec     55. 68.100 / 55. 68.100
  libavformat    55. 44.100 / 55. 44.100
  libavdevice    55. 13.101 / 55. 13.101
  libavfilter     4.  9.100 /  4.  9.100
  libswscale      2.  6.100 /  2.  6.100
  libswresample   0. 19.100 /  0. 19.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'v210_cant_be_jpeg2000d.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 512
    compatible_brands: qt
    encoder         : Lavf55.44.100
  Duration: 00:00:01.02, start: 0.021333, bitrate: 219267 kb/s
    Stream #0:0(eng): Video: v210 (v210 / 0x30313276), yuv422p10le, 720x486, 223725 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 30k tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc55.68.100 v210
    Stream #0:1(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 123 kb/s (default)
    Metadata:
      handler_name    : DataHandler
[crc @ 0x37f0a00] Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead.
Output #0, crc, to 'pipe:':
  Metadata:
    major_brand     : qt
    minor_version   : 512
    compatible_brands: qt
    encoder         : Lavf55.44.100
    Stream #0:0(eng): Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le, 720x486, q=2-31, 200 kb/s, 29.97 fps, 29.97 tbn, 29.97 tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc55.68.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (v210 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
CRC=0x69c0112a
frame=   30 fps=0.0 q=0.0 Lsize=       0kB time=00:00:01.03 bitrate=   0.1kbits/s
video:41006kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
$ ffmpeg -vsync 0 -i v210_cant_be_jpeg2000d.mov -an -vcodec libopenjpeg out.mov
$ ffmpeg -vsync 0 -i out.mov -f crc -
ffmpeg version N-64232-g932be89 Copyright (c) 2000-2014 the FFmpeg developers
  built on Jun 26 2014 20:31:10 with gcc 4.7 (SUSE Linux)
  configuration: --enable-libopenjpeg --disable-decoder=jpeg2000
  libavutil      52. 90.101 / 52. 90.101
  libavcodec     55. 68.100 / 55. 68.100
  libavformat    55. 44.100 / 55. 44.100
  libavdevice    55. 13.101 / 55. 13.101
  libavfilter     4.  9.100 /  4.  9.100
  libswscale      2.  6.100 /  2.  6.100
  libswresample   0. 19.100 /  0. 19.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'out.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 512
    compatible_brands: qt
    encoder         : Lavf55.44.100
  Duration: 00:00:01.00, start: 0.033000, bitrate: 107620 kb/s
    Stream #0:0(eng): Video: jpeg2000 (mjp2 / 0x32706A6D), yuv422p10le, 720x486, 107612 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 30k tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc55.68.100 libopenjpeg
[crc @ 0x2ba1500] Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead.
Output #0, crc, to 'pipe:':
  Metadata:
    major_brand     : qt
    minor_version   : 512
    compatible_brands: qt
    encoder         : Lavf55.44.100
    Stream #0:0(eng): Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le, 720x486, q=2-31, 200 kb/s, 29.97 fps, 29.97 tbn, 29.97 tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc55.68.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (jpeg2000 (libopenjpeg) -> rawvideo (native))
Press [q] to stop, [?] for help
CRC=0x69c0112a
frame=   30 fps=0.0 q=0.0 Lsize=       0kB time=00:00:01.00 bitrate=   0.1kbits/s
video:41006kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

-f framecrc is also identical for both files here (apart from the timestamp issue).

Could this be an issue with your openjpeg version (or compilation)?

comment:2 Changed 4 years ago by dericed

--disable-decoder=jpeg2000 :)

When I replicate your steps I get CRC=0x1d97037f as the output of the j2k version. This is on my Mac as well as another Ubuntu computer. On both I've been using https://openjpeg.googlecode.com/files/openjpeg-1.5.1.tar.gz whereas you've been using openjpeg-1.5.0-2.7.1.x86_64. To proceed I'll see if I can replicate with openjpeg alone and then send the ticket, their way.

And yes I've seen the QuickTime? VFR issues in other tickets.

btw here is a copy of my out.mov with the glitch issue frame 17 http://dericed.com/samples/j2k_error_frame17_out.mov

comment:3 Changed 4 years ago by cehoyos

Works fine here with your file:

$ ffmpeg -vsync 0 -i j2k_error_frame17_out.mov -f crc -
ffmpeg version N-64247-gb9bfd88 Copyright (c) 2000-2014 the FFmpeg developers
  built on Jun 26 2014 23:08:12 with gcc 4.7 (SUSE Linux)
  configuration: --enable-libopenjpeg --disable-decoder=jpeg2000
  libavutil      52. 90.101 / 52. 90.101
  libavcodec     55. 68.100 / 55. 68.100
  libavformat    55. 44.100 / 55. 44.100
  libavdevice    55. 13.101 / 55. 13.101
  libavfilter     4.  9.100 /  4.  9.100
  libswscale      2.  6.100 /  2.  6.100
  libswresample   0. 19.100 /  0. 19.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'j2k_error_frame17_out.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 512
    compatible_brands: qt
    encoder         : Lavf55.44.100
  Duration: 00:00:01.00, start: 0.033000, bitrate: 107620 kb/s
    Stream #0:0(eng): Video: jpeg2000 (mjp2 / 0x32706A6D), yuv422p10le, 720x486, 107612 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 30k tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc55.67.100 libopenjpeg
[crc @ 0x2953500] Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead.
Output #0, crc, to 'pipe:':
  Metadata:
    major_brand     : qt
    minor_version   : 512
    compatible_brands: qt
    encoder         : Lavf55.44.100
    Stream #0:0(eng): Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le, 720x486, q=2-31, 200 kb/s, 29.97 fps, 29.97 tbn, 29.97 tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc55.68.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (jpeg2000 (libopenjpeg) -> rawvideo (native))
Press [q] to stop, [?] for help
CRC=0x69c0112a
frame=   30 fps=0.0 q=0.0 Lsize=       0kB time=00:00:01.00 bitrate=   0.1kbits/s
video:41006kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

Changed 4 years ago by cehoyos

comment:4 Changed 4 years ago by cehoyos

Can you reproduce your issue with the attached frame?

comment:5 Changed 4 years ago by dericed

Yes. If I view in QuickTime? it seems okay, but if I use ffplay -vcodec libopenjpeg or the command below the output has the same chroma plane issue.

ffmpeg -vcodec libopenjpeg -i out17.j2k out17.tiff         
ffmpeg version N-64075-g47c8d1e Copyright (c) 2000-2014 the FFmpeg developers
  built on Jun 19 2014 10:18:25 with Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-nonfree --enable-hardcoded-tables --enable-avresample --enable-vda --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libfaac --enable-libmp3lame --enable-libxvid --enable-libfreetype --enable-ffplay --enable-libopenjpeg --extra-cflags='-I/usr/local/Cellar/openjpeg/1.5.1_1/include/openjpeg-1.5 '
  libavutil      52. 89.100 / 52. 89.100
  libavcodec     55. 67.100 / 55. 67.100
  libavformat    55. 44.100 / 55. 44.100
  libavdevice    55. 13.101 / 55. 13.101
  libavfilter     4.  8.100 /  4.  8.100
  libavresample   1.  3.  0 /  1.  3.  0
  libswscale      2.  6.100 /  2.  6.100
  libswresample   0. 19.100 /  0. 19.100
  libpostproc    52.  3.100 / 52.  3.100
Input #0, image2, from 'out17.j2k':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: jpeg2000, yuv422p10le, 720x486, 25 tbr, 25 tbn, 25 tbc
[image2 @ 0x7feb5900b000] Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead.
Output #0, image2, to 'out17.tiff':
  Metadata:
    encoder         : Lavf55.44.100
    Stream #0:0: Video: tiff, rgb48le, 720x486, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc55.67.100 tiff
Stream mapping:
  Stream #0:0 -> #0:0 (jpeg2000 (libopenjpeg) -> tiff (native))
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A    
video:2055kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

Changed 4 years ago by dericed

comment:6 Changed 4 years ago by cehoyos

I can reproduce your problem with libopenjpeg-1.5.1
Is there any indication that this is a problem that can be fixed within FFmpeg?

comment:7 follow-up: Changed 4 years ago by dericed

No, I don't think this can be fixed in ffmpeg but is with openjpeg. There is an associated request here http://trac.ffmpeg.org/ticket/2866 to request an option for ffmpeg to read what it is encoding to verify losslessness, similar to the --verify option in the flac utility. Thanks to framemd5 I was able to find it.

I tested the j2k you sent with openjpeg 2.1.0 with opj_decompress -i ~/Downloads/out17.j2k -o image.tif, which decoded correctly. I then tested with 1.5.1 and got

j2k_to_image -i out17.j2k -o image.png

[ERROR] 00000000: expected a marker instead of 0
ERROR -> j2k_to_image: failed to decode image!

Next I'm off to see if ffmpeg will build with openjpeg 2.1.0.

comment:8 in reply to: ↑ 7 Changed 4 years ago by cehoyos

  • Resolution set to invalid
  • Status changed from new to closed

Replying to dericed:

Next I'm off to see if ffmpeg will build with openjpeg 2.1.0.

See ticket #2016

comment:9 Changed 4 years ago by cehoyos

The issue is not reproducible with openjpeg 2.1.0.

Note: See TracTickets for help on using tickets.