Opened 10 years ago

Closed 10 years ago

Last modified 5 years ago

#3874 closed defect (fixed)

tiff encoder creates broken odd width yuv files

Reported by: ami_stuff Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: tif
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

sample from ticket #1178

https://trac.ffmpeg.org/raw-attachment/ticket/1178/out.tif

C:\>ffmpegx -i out.tif out.bmp
ffmpeg version N-65738-g3c197442 Copyright (c) 2000-2014 the FFmpeg developers
  built on Aug 19 2014 02:46:13 with gcc 4.5.0 (GCC) 20100414 (Fedora MinGW 4.5.
0-1.fc14)
  configuration: --prefix=/var/www/users/research/ffmpeg/snapshots/build --arch=
x86 --target-os=mingw32 --cross-prefix=i686-pc-mingw32- --cc='ccache i686-pc-min
gw32-gcc' --enable-pthreads --enable-memalign-hack --enable-runtime-cpudetect --
enable-cross-compile --enable-static --disable-shared --extra-libs='-lws2_32 -lw
inmm -lpthread' --extra-cflags='--static -I/var/www/users/research/ffmpeg/snapsh
ots/build/include' --extra-ldflags='-static -L/var/www/users/research/ffmpeg/sna
pshots/build/lib' --enable-bzlib --enable-zlib --enable-gpl --enable-version3 --
enable-nonfree --enable-libx264 --enable-libspeex --enable-libtheora --enable-li
bvorbis --enable-libfaac --enable-libxvid --enable-libopencore-amrnb --enable-li
bopencore-amrwb --enable-libmp3lame --enable-libfreetype --enable-libvpx --disab
le-decoder=libvpx
  libavutil      54.  5.100 / 54.  5.100
  libavcodec     56.  0.101 / 56.  0.101
  libavformat    56.  1.100 / 56.  1.100
  libavdevice    56.  0.100 / 56.  0.100
  libavfilter     5.  0.100 /  5.  0.100
  libswscale      3.  0.100 /  3.  0.100
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  0.100 / 53.  0.100
[tiff @ 0x230b2c0] Copy went out of bounds
Input #0, image2, from 'out.tif':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: tiff, yuv410p, 323x240 [SAR 1:1 DAR 323:240], 25 tbr, 25
 tbn, 25 tbc
Output #0, image2, to 'out.bmp':
  Metadata:
    encoder         : Lavf56.1.100
    Stream #0:0: Video: bmp, bgr24, 323x240 [SAR 1:1 DAR 323:240], q=2-31, 200 k
b/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc56.0.101 bmp
Stream mapping:
  Stream #0:0 -> #0:0 (tiff (native) -> bmp (native))
Press [q] to stop, [?] for help
[tiff @ 0x23075e0] Copy went out of bounds
frame=    1 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
video:228kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing o
verhead: unknown

Attachments (3)

rgb.tif (137.5 KB ) - added by jamal 10 years ago.
libtiff "tiff2rgba" output
0001-tiffenc-properly-compute-packet-size.patch (1.4 KB ) - added by Christophe 10 years ago.
Tentative fix
yuv.tiff (86.0 KB ) - added by Christophe 10 years ago.
YUV 4:1:0 generated with rgb2ycbcr

Download all attachments as: .zip

Change History (25)

by jamal, 10 years ago

Attachment: rgb.tif added

libtiff "tiff2rgba" output

comment:1 by jamal, 10 years ago

Component: undeterminedavcodec

The file seems to be broken.
One of libtiff's tools also has trouble decoding it, and the output has artifacts after giving several "Discarding 9 bytes to avoid buffer overrun" warnings (see attached resulting file, which decodes fine with ffmpeg git master).

Fun thing is, this was created by an old ffmpeg build (pre 0.11 it seems), so the encoder was probably at fault here.

in reply to:  1 comment:2 by ami_stuff, 10 years ago

Replying to jamal:

Fun thing is, this was created by an old ffmpeg build (pre 0.11 it seems), so the encoder was probably at fault here.

Thanks for looking at it.

It seems that encoder is still broken, test this:

https://trac.ffmpeg.org/raw-attachment/ticket/1178/yuv410p.avi

ffmpeg -i yuv410p.avi out.tif

out.tif will fail to decode

comment:3 by jamal, 10 years ago

Keywords: tiff added
Status: newopen
Summary: tiff: yuv with odd width fails to decodetiff encoder creates broken odd width yuv files with packbits compression
Version: unspecifiedgit-master

Seems to be packbits encoding only.

ffmpeg -i yuv410p.avi -compression_algo raw raw.tif
ffmpeg -i yuv410p.avi -compression_algo lzw lzw.tif
ffmpeg -i yuv410p.avi -compression_algo deflate deflate.tif

All three work fine.

comment:4 by jamal, 10 years ago

Ok, they "work", with emphasis on the quotes.
They don't trigger errors/warnings during decoding like packbits does, but the output is completely broken with ffmpeg, and similar to the attached file with tiff2rgb, so our decoder may be wrong as well.

in reply to:  3 comment:5 by ami_stuff, 10 years ago

Replying to jamal:

Seems to be packbits encoding only.

ffmpeg -i yuv410p.avi -compression_algo raw raw.tif
ffmpeg -i yuv410p.avi -compression_algo lzw lzw.tif
ffmpeg -i yuv410p.avi -compression_algo deflate deflate.tif

All three work fine.

These looks broken as well while decoded with IrfanView.

comment:6 by Carl Eugen Hoyos, 10 years ago

Keywords: tif added; tiff removed

comment:7 by jamal, 10 years ago

Summary: tiff encoder creates broken odd width yuv files with packbits compressiontiff encoder creates broken odd width yuv files

comment:8 by Christophe, 10 years ago

Does the bug still exist? I don't know what could have fixed this but:

  • ffmpeg and tiff2rgba decode fine all outputs of ffmpeg, whatever the compression scheme;
  • The compression scheme seems to be applied at first glance (varying size for output tiff file);
  • Some software (e.g. Xnview) seem to decode incorrectly the files (black square inserted corresponding to the sampling block, then of course lines get shifted) anyway, so they should not be trusted.

in reply to:  8 comment:9 by ami_stuff, 10 years ago

Replying to kurosu:

Does the bug still exist? I don't know what could have fixed this but:

  • ffmpeg and tiff2rgba decode fine all outputs of ffmpeg, whatever the compression scheme;
  • The compression scheme seems to be applied at first glance (varying size for output tiff file);
  • Some software (e.g. Xnview) seem to decode incorrectly the files (black square inserted corresponding to the sampling block, then of course lines get shifted) anyway, so they should not be trusted.

I see the same broken output (green artefacts) as before with

ffmpeg -i yuv410p.avi out.tif

by Christophe, 10 years ago

Tentative fix

comment:10 by Christophe, 10 years ago

Err, yes, I had a patch I thought fixed nothing applied locally, but it seems to actually fix things for me. Could you test the patch I've attached to this bug report?

in reply to:  10 ; comment:11 by ami_stuff, 10 years ago

Replying to kurosu:

Err, yes, I had a patch I thought fixed nothing applied locally, but it seems to actually fix things for me. Could you test the patch I've attached to this bug report?

Applied, but effect is the same - green artefact when viewing output tiff with IrfanView.

in reply to:  11 ; comment:12 by Christophe, 10 years ago

Keywords: tiffenc added

Replying to ami_stuff:

Applied, but effect is the same - green artefact when viewing output tiff with IrfanView.

Don't test with IrfanView, or Gimp, or XnView. I've just tested, these ones have broken output, but libtiff (eg tiff2rgba) and ffmpeg (with the patch applied) do not. I suspect most softwares do not abide by the rule set in libtiff, encode extended image data if need be.

Second, are the softwares you use able to encode YUV with chroma subsampling? And is their output decodable by libtiff? By themselves? Are they able to decode libtiff output?

If you do have a file not produced by ffmpeg but that decodes well in both libtiff and some software, that would be interesting.

Last edited 10 years ago by Christophe (previous) (diff)

comment:13 by ami_stuff, 10 years ago

Ok, so I downloaded some old ffmpeg autobuild from august and tiff2rgba decodes generated tiff file just fine
(I get no error messages and output looks correctly)

TIFF file generated with your patch decodes ok as well with tiff2rgba.

The only problem happens with other software like ImageMagick and the rest which, as I understand, you belive is at fauly in some way. If this is true then this ticket can be closed.

Second, are the softwares you use able to encode YUV with chroma subsampling? And is their output decodable by libtiff? By themselves? Are they able to decode libtiff output?

I don't know any software (other than ffmpeg) which can generate YUV TIFFs.

C:\>ffmpeg -i yuv410p.avi out.tif
ffmpeg version N-65738-g3c197442 Copyright (c) 2000-2014 the FFmpeg developers
  built on Aug 18 2014 22:02:08 with gcc 4.8.3 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av
isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --
enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-lib
modplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrw
b --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinge
r --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --en
able-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis
 --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-
libx265 --enable-libxavs --enable-libxvid --enable-decklink --enable-zlib
  libavutil      54.  5.100 / 54.  5.100
  libavcodec     56.  0.101 / 56.  0.101
  libavformat    56.  1.100 / 56.  1.100
  libavdevice    56.  0.100 / 56.  0.100
  libavfilter     5.  0.100 /  5.  0.100
  libswscale      3.  0.100 /  3.  0.100
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  0.100 / 53.  0.100
Input #0, avi, from 'yuv410p.avi':
  Metadata:
    encoder         : Lavf54.3.100
  Duration: 00:00:00.04, start: 0.000000, bitrate: 9856 kb/s
    Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv410p, 323x240, 25 fps, 25 t
br, 25 tbn, 25 tbc
Output #0, image2, to 'out.tif':
  Metadata:
    encoder         : Lavf56.1.100
    Stream #0:0: Video: tiff, yuv410p, 323x240, q=2-31, 200 kb/s, 25 fps, 25 tbn
, 25 tbc
    Metadata:
      encoder         : Lavc56.0.101 tiff
Stream mapping:
  Stream #0:0 -> #0:0 (ffv1 (native) -> tiff (native))
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
video:86kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing ov
erhead: unknown
C:\>tiff2rgba out.tif out2.tif
Last edited 10 years ago by ami_stuff (previous) (diff)

by Christophe, 10 years ago

Attachment: yuv.tiff added

YUV 4:1:0 generated with rgb2ycbcr

in reply to:  13 ; comment:15 by Christophe, 10 years ago

Replying to ami_stuff:

The only problem happens with other software like ImageMagick and the rest which, as I understand, you belive is at fauly in some way. If this is true then this ticket can be closed.

Yes, try the attached yuv.tiff generated by:
ffmpeg -i lena.pnm -s 323x240 lena.tiff
rgb2ycbcr -c none -h 4 -v 4 lena.tiff yuv.tiff

Gimp displays green garbages at the bottom of stripes I suspect to be a multiple of 4 in height. XnView shows staircases. ffmpeg decoded output is fine.

I don't know any software (other than ffmpeg) which can generate YUV TIFFs.

Please test them with the attached file or one generated following your preferences.

Regarding the yuv data, the real issue is rather the sampling. See the h/v parameters to rgbycbcr:
http://www.remotesensing.org/libtiff/man/rgb2ycbcr.1.html

comment:16 by Christophe, 10 years ago

Note, ffmpeg encodes fine only with the attached patch.

I'll edit this patch before sending it in case you agree it's at least a partial fix.

in reply to:  16 comment:17 by ami_stuff, 10 years ago

Replying to kurosu:

Note, ffmpeg encodes fine only with the attached patch.

I'll edit this patch before sending it in case you agree it's at least a partial fix.

Please go ahead.

in reply to:  15 ; comment:18 by ami_stuff, 10 years ago

Replying to kurosu:

Yes, try the attached yuv.tiff generated by:
ffmpeg -i lena.pnm -s 323x240 lena.tiff
rgb2ycbcr -c none -h 4 -v 4 lena.tiff yuv.tiff

Gimp displays green garbages at the bottom of stripes I suspect to be a multiple of 4 in height. XnView shows staircases. ffmpeg decoded output is fine.

I don't know any software (other than ffmpeg) which can generate YUV TIFFs.

Please test them with the attached file or one generated following your preferences.

I can confirm that there are visible the same artefacts like with tiff file generated by ffmpeg.

in reply to:  18 ; comment:19 by Christophe, 10 years ago

Replying to ami_stuff:

I can confirm that there are visible the same artefacts like with tiff file generated by ffmpeg.

So if I'm understanding it well, once the patch is applied, we will be able to summarize the new situation as "non-ffmpeg decoder bugs, the spec-compliant ones (including ffmpeg) are fine".

in reply to:  19 comment:20 by ami_stuff, 10 years ago

Replying to kurosu:

Replying to ami_stuff:

I can confirm that there are visible the same artefacts like with tiff file generated by ffmpeg.

So if I'm understanding it well, once the patch is applied, we will be able to summarize the new situation as "non-ffmpeg decoder bugs, the spec-compliant ones (including ffmpeg) are fine".

Yes, I think so.

comment:21 by Michael Niedermayer, 10 years ago

Resolution: fixed
Status: openclosed

(if i understand correctly) Fixed by Christophe Gisquet in 0e8bfd3c934768f9812dd20d71fa4709de54186d

comment:22 by Carl Eugen Hoyos, 5 years ago

Keywords: tiffenc removed
Note: See TracTickets for help on using tickets.