tiff encoder creates broken odd width yuv files

[ami_stuff]

sample from ticket #1178

​https://trac.ffmpeg.org/raw-attachment/ticket/1178/out.tif

C:\>ffmpegx -i out.tif out.bmp
ffmpeg version N-65738-g3c197442 Copyright (c) 2000-2014 the FFmpeg developers
  built on Aug 19 2014 02:46:13 with gcc 4.5.0 (GCC) 20100414 (Fedora MinGW 4.5.
0-1.fc14)
  configuration: --prefix=/var/www/users/research/ffmpeg/snapshots/build --arch=
x86 --target-os=mingw32 --cross-prefix=i686-pc-mingw32- --cc='ccache i686-pc-min
gw32-gcc' --enable-pthreads --enable-memalign-hack --enable-runtime-cpudetect --
enable-cross-compile --enable-static --disable-shared --extra-libs='-lws2_32 -lw
inmm -lpthread' --extra-cflags='--static -I/var/www/users/research/ffmpeg/snapsh
ots/build/include' --extra-ldflags='-static -L/var/www/users/research/ffmpeg/sna
pshots/build/lib' --enable-bzlib --enable-zlib --enable-gpl --enable-version3 --
enable-nonfree --enable-libx264 --enable-libspeex --enable-libtheora --enable-li
bvorbis --enable-libfaac --enable-libxvid --enable-libopencore-amrnb --enable-li
bopencore-amrwb --enable-libmp3lame --enable-libfreetype --enable-libvpx --disab
le-decoder=libvpx
  libavutil      54.  5.100 / 54.  5.100
  libavcodec     56.  0.101 / 56.  0.101
  libavformat    56.  1.100 / 56.  1.100
  libavdevice    56.  0.100 / 56.  0.100
  libavfilter     5.  0.100 /  5.  0.100
  libswscale      3.  0.100 /  3.  0.100
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  0.100 / 53.  0.100
[tiff @ 0x230b2c0] Copy went out of bounds
Input #0, image2, from 'out.tif':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: tiff, yuv410p, 323x240 [SAR 1:1 DAR 323:240], 25 tbr, 25
 tbn, 25 tbc
Output #0, image2, to 'out.bmp':
  Metadata:
    encoder         : Lavf56.1.100
    Stream #0:0: Video: bmp, bgr24, 323x240 [SAR 1:1 DAR 323:240], q=2-31, 200 k
b/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc56.0.101 bmp
Stream mapping:
  Stream #0:0 -> #0:0 (tiff (native) -> bmp (native))
Press [q] to stop, [?] for help
[tiff @ 0x23075e0] Copy went out of bounds
frame=    1 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
video:228kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing o
verhead: unknown

[jamal]

libtiff "tiff2rgba" output

[jamal]

The file seems to be broken.
One of libtiff's tools also has trouble decoding it, and the output has artifacts after giving several "Discarding 9 bytes to avoid buffer overrun" warnings (see attached resulting file, which decodes fine with ffmpeg git master).

Fun thing is, this was created by an old ffmpeg build (pre 0.11 it seems), so the encoder was probably at fault here.

[ami_stuff]

Replying to jamal:

Fun thing is, this was created by an old ffmpeg build (pre 0.11 it seems), so the encoder was probably at fault here.

Thanks for looking at it.

It seems that encoder is still broken, test this:

​https://trac.ffmpeg.org/raw-attachment/ticket/1178/yuv410p.avi

ffmpeg -i yuv410p.avi out.tif

out.tif will fail to decode

[jamal]

Seems to be packbits encoding only.

ffmpeg -i yuv410p.avi -compression_algo raw raw.tif
ffmpeg -i yuv410p.avi -compression_algo lzw lzw.tif
ffmpeg -i yuv410p.avi -compression_algo deflate deflate.tif

All three work fine.

[jamal]

Ok, they "work", with emphasis on the quotes.
They don't trigger errors/warnings during decoding like packbits does, but the output is completely broken with ffmpeg, and similar to the attached file with tiff2rgb, so our decoder may be wrong as well.

[ami_stuff]

Replying to jamal:

Seems to be packbits encoding only.

ffmpeg -i yuv410p.avi -compression_algo raw raw.tif
ffmpeg -i yuv410p.avi -compression_algo lzw lzw.tif
ffmpeg -i yuv410p.avi -compression_algo deflate deflate.tif

All three work fine.

These looks broken as well while decoded with IrfanView.

[Christophe]

Does the bug still exist? I don't know what could have fixed this but:

• ffmpeg and tiff2rgba decode fine all outputs of ffmpeg, whatever the compression scheme;
• The compression scheme seems to be applied at first glance (varying size for output tiff file);
• Some software (e.g. Xnview) seem to decode incorrectly the files (black square inserted corresponding to the sampling block, then of course lines get shifted) anyway, so they should not be trusted.

[ami_stuff]

Replying to kurosu:

Does the bug still exist? I don't know what could have fixed this but:

• ffmpeg and tiff2rgba decode fine all outputs of ffmpeg, whatever the compression scheme;
• The compression scheme seems to be applied at first glance (varying size for output tiff file);
• Some software (e.g. Xnview) seem to decode incorrectly the files (black square inserted corresponding to the sampling block, then of course lines get shifted) anyway, so they should not be trusted.

I see the same broken output (green artefacts) as before with

ffmpeg -i yuv410p.avi out.tif

[Christophe]

Tentative fix

[Christophe]

Err, yes, I had a patch I thought fixed nothing applied locally, but it seems to actually fix things for me. Could you test the patch I've attached to this bug report?

[ami_stuff]

Replying to kurosu:

Err, yes, I had a patch I thought fixed nothing applied locally, but it seems to actually fix things for me. Could you test the patch I've attached to this bug report?

Applied, but effect is the same - green artefact when viewing output tiff with IrfanView.

[Christophe]

Replying to ami_stuff:

Applied, but effect is the same - green artefact when viewing output tiff with IrfanView.

Don't test with IrfanView, or Gimp, or XnView. I've just tested, these ones have broken output, but libtiff (eg tiff2rgba) and ffmpeg (with the patch applied) do not. I suspect most softwares do not abide by the rule set in libtiff, encode extended image data if need be.

Second, are the softwares you use able to encode YUV with chroma subsampling? And is their output decodable by libtiff? By themselves? Are they able to decode libtiff output?

If you do have a file not produced by ffmpeg but that decodes well in both libtiff and some software, that would be interesting.

[ami_stuff]

Ok, so I downloaded some old ffmpeg autobuild from august and tiff2rgba decodes generated tiff file just fine
(I get no error messages and output looks correctly)

TIFF file generated with your patch decodes ok as well with tiff2rgba.

The only problem happens with other software like ImageMagick and the rest which, as I understand, you belive is at fauly in some way. If this is true then this ticket can be closed.

Second, are the softwares you use able to encode YUV with chroma subsampling? And is their output decodable by libtiff? By themselves? Are they able to decode libtiff output?

I don't know any software (other than ffmpeg) which can generate YUV TIFFs.

C:\>ffmpeg -i yuv410p.avi out.tif
ffmpeg version N-65738-g3c197442 Copyright (c) 2000-2014 the FFmpeg developers
  built on Aug 18 2014 22:02:08 with gcc 4.8.3 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av
isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --
enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-lib
modplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrw
b --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinge
r --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --en
able-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis
 --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-
libx265 --enable-libxavs --enable-libxvid --enable-decklink --enable-zlib
  libavutil      54.  5.100 / 54.  5.100
  libavcodec     56.  0.101 / 56.  0.101
  libavformat    56.  1.100 / 56.  1.100
  libavdevice    56.  0.100 / 56.  0.100
  libavfilter     5.  0.100 /  5.  0.100
  libswscale      3.  0.100 /  3.  0.100
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  0.100 / 53.  0.100
Input #0, avi, from 'yuv410p.avi':
  Metadata:
    encoder         : Lavf54.3.100
  Duration: 00:00:00.04, start: 0.000000, bitrate: 9856 kb/s
    Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv410p, 323x240, 25 fps, 25 t
br, 25 tbn, 25 tbc
Output #0, image2, to 'out.tif':
  Metadata:
    encoder         : Lavf56.1.100
    Stream #0:0: Video: tiff, yuv410p, 323x240, q=2-31, 200 kb/s, 25 fps, 25 tbn
, 25 tbc
    Metadata:
      encoder         : Lavc56.0.101 tiff
Stream mapping:
  Stream #0:0 -> #0:0 (ffv1 (native) -> tiff (native))
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
video:86kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing ov
erhead: unknown

C:\>tiff2rgba out.tif out2.tif

[ami_stuff]

Here is some YUV sample:

​http://www.steves-digicams.com/qv3000/samples/08233627.tif

​http://www.steves-digicams.com/qv3000_samples.html

[Christophe]

YUV 4:1:0 generated with rgb2ycbcr

[Christophe]

Replying to ami_stuff:

The only problem happens with other software like ImageMagick and the rest which, as I understand, you belive is at fauly in some way. If this is true then this ticket can be closed.

Yes, try the attached yuv.tiff generated by:
ffmpeg -i lena.pnm -s 323x240 lena.tiff
rgb2ycbcr -c none -h 4 -v 4 lena.tiff yuv.tiff

Gimp displays green garbages at the bottom of stripes I suspect to be a multiple of 4 in height. XnView shows staircases. ffmpeg decoded output is fine.

I don't know any software (other than ffmpeg) which can generate YUV TIFFs.

Please test them with the attached file or one generated following your preferences.

Regarding the yuv data, the real issue is rather the sampling. See the h/v parameters to rgbycbcr:
​http://www.remotesensing.org/libtiff/man/rgb2ycbcr.1.html

[Christophe]

Note, ffmpeg encodes fine only with the attached patch.

I'll edit this patch before sending it in case you agree it's at least a partial fix.

[ami_stuff]

Replying to kurosu:

Note, ffmpeg encodes fine only with the attached patch.

I'll edit this patch before sending it in case you agree it's at least a partial fix.

Please go ahead.

[ami_stuff]

Replying to kurosu:

Yes, try the attached yuv.tiff generated by:
ffmpeg -i lena.pnm -s 323x240 lena.tiff
rgb2ycbcr -c none -h 4 -v 4 lena.tiff yuv.tiff

Gimp displays green garbages at the bottom of stripes I suspect to be a multiple of 4 in height. XnView shows staircases. ffmpeg decoded output is fine.

I don't know any software (other than ffmpeg) which can generate YUV TIFFs.

Please test them with the attached file or one generated following your preferences.

I can confirm that there are visible the same artefacts like with tiff file generated by ffmpeg.

[Christophe]

Replying to ami_stuff:

I can confirm that there are visible the same artefacts like with tiff file generated by ffmpeg.

So if I'm understanding it well, once the patch is applied, we will be able to summarize the new situation as "non-ffmpeg decoder bugs, the spec-compliant ones (including ffmpeg) are fine".

[ami_stuff]

Replying to kurosu:

Replying to ami_stuff:

I can confirm that there are visible the same artefacts like with tiff file generated by ffmpeg.

So if I'm understanding it well, once the patch is applied, we will be able to summarize the new situation as "non-ffmpeg decoder bugs, the spec-compliant ones (including ffmpeg) are fine".

Yes, I think so.

[Michael Niedermayer]

(if i understand correctly) Fixed by Christophe Gisquet in 0e8bfd3c934768f9812dd20d71fa4709de54186d