Opened 5 years ago

Closed 5 years ago

#3874 closed defect (fixed)

tiff encoder creates broken odd width yuv files

Reported by: ami_stuff Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: tif tiffenc
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

sample from ticket #1178

https://trac.ffmpeg.org/raw-attachment/ticket/1178/out.tif

C:\>ffmpegx -i out.tif out.bmp
ffmpeg version N-65738-g3c197442 Copyright (c) 2000-2014 the FFmpeg developers
  built on Aug 19 2014 02:46:13 with gcc 4.5.0 (GCC) 20100414 (Fedora MinGW 4.5.
0-1.fc14)
  configuration: --prefix=/var/www/users/research/ffmpeg/snapshots/build --arch=
x86 --target-os=mingw32 --cross-prefix=i686-pc-mingw32- --cc='ccache i686-pc-min
gw32-gcc' --enable-pthreads --enable-memalign-hack --enable-runtime-cpudetect --
enable-cross-compile --enable-static --disable-shared --extra-libs='-lws2_32 -lw
inmm -lpthread' --extra-cflags='--static -I/var/www/users/research/ffmpeg/snapsh
ots/build/include' --extra-ldflags='-static -L/var/www/users/research/ffmpeg/sna
pshots/build/lib' --enable-bzlib --enable-zlib --enable-gpl --enable-version3 --
enable-nonfree --enable-libx264 --enable-libspeex --enable-libtheora --enable-li
bvorbis --enable-libfaac --enable-libxvid --enable-libopencore-amrnb --enable-li
bopencore-amrwb --enable-libmp3lame --enable-libfreetype --enable-libvpx --disab
le-decoder=libvpx
  libavutil      54.  5.100 / 54.  5.100
  libavcodec     56.  0.101 / 56.  0.101
  libavformat    56.  1.100 / 56.  1.100
  libavdevice    56.  0.100 / 56.  0.100
  libavfilter     5.  0.100 /  5.  0.100
  libswscale      3.  0.100 /  3.  0.100
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  0.100 / 53.  0.100
[tiff @ 0x230b2c0] Copy went out of bounds
Input #0, image2, from 'out.tif':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: tiff, yuv410p, 323x240 [SAR 1:1 DAR 323:240], 25 tbr, 25
 tbn, 25 tbc
Output #0, image2, to 'out.bmp':
  Metadata:
    encoder         : Lavf56.1.100
    Stream #0:0: Video: bmp, bgr24, 323x240 [SAR 1:1 DAR 323:240], q=2-31, 200 k
b/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc56.0.101 bmp
Stream mapping:
  Stream #0:0 -> #0:0 (tiff (native) -> bmp (native))
Press [q] to stop, [?] for help
[tiff @ 0x23075e0] Copy went out of bounds
frame=    1 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
video:228kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing o
verhead: unknown

Attachments (3)

rgb.tif (137.5 KB) - added by jamal 5 years ago.
libtiff "tiff2rgba" output
0001-tiffenc-properly-compute-packet-size.patch (1.4 KB) - added by kurosu 5 years ago.
Tentative fix
yuv.tiff (86.0 KB) - added by kurosu 5 years ago.
YUV 4:1:0 generated with rgb2ycbcr

Download all attachments as: .zip

Change History (24)

Changed 5 years ago by jamal

libtiff "tiff2rgba" output

comment:1 follow-up: Changed 5 years ago by jamal

  • Component changed from undetermined to avcodec

The file seems to be broken.
One of libtiff's tools also has trouble decoding it, and the output has artifacts after giving several "Discarding 9 bytes to avoid buffer overrun" warnings (see attached resulting file, which decodes fine with ffmpeg git master).

Fun thing is, this was created by an old ffmpeg build (pre 0.11 it seems), so the encoder was probably at fault here.

comment:2 in reply to: ↑ 1 Changed 5 years ago by ami_stuff

Replying to jamal:

Fun thing is, this was created by an old ffmpeg build (pre 0.11 it seems), so the encoder was probably at fault here.

Thanks for looking at it.

It seems that encoder is still broken, test this:

https://trac.ffmpeg.org/raw-attachment/ticket/1178/yuv410p.avi

ffmpeg -i yuv410p.avi out.tif

out.tif will fail to decode

comment:3 follow-up: Changed 5 years ago by jamal

  • Keywords tiff added
  • Status changed from new to open
  • Summary changed from tiff: yuv with odd width fails to decode to tiff encoder creates broken odd width yuv files with packbits compression
  • Version changed from unspecified to git-master

Seems to be packbits encoding only.

ffmpeg -i yuv410p.avi -compression_algo raw raw.tif
ffmpeg -i yuv410p.avi -compression_algo lzw lzw.tif
ffmpeg -i yuv410p.avi -compression_algo deflate deflate.tif

All three work fine.

comment:4 Changed 5 years ago by jamal

Ok, they "work", with emphasis on the quotes.
They don't trigger errors/warnings during decoding like packbits does, but the output is completely broken with ffmpeg, and similar to the attached file with tiff2rgb, so our decoder may be wrong as well.

comment:5 in reply to: ↑ 3 Changed 5 years ago by ami_stuff

Replying to jamal:

Seems to be packbits encoding only.

ffmpeg -i yuv410p.avi -compression_algo raw raw.tif
ffmpeg -i yuv410p.avi -compression_algo lzw lzw.tif
ffmpeg -i yuv410p.avi -compression_algo deflate deflate.tif

All three work fine.

These looks broken as well while decoded with IrfanView?.

comment:6 Changed 5 years ago by cehoyos

  • Keywords tif added; tiff removed

comment:7 Changed 5 years ago by jamal

  • Summary changed from tiff encoder creates broken odd width yuv files with packbits compression to tiff encoder creates broken odd width yuv files

comment:8 follow-up: Changed 5 years ago by kurosu

Does the bug still exist? I don't know what could have fixed this but:

  • ffmpeg and tiff2rgba decode fine all outputs of ffmpeg, whatever the compression scheme;
  • The compression scheme seems to be applied at first glance (varying size for output tiff file);
  • Some software (e.g. Xnview) seem to decode incorrectly the files (black square inserted corresponding to the sampling block, then of course lines get shifted) anyway, so they should not be trusted.

comment:9 in reply to: ↑ 8 Changed 5 years ago by ami_stuff

Replying to kurosu:

Does the bug still exist? I don't know what could have fixed this but:

  • ffmpeg and tiff2rgba decode fine all outputs of ffmpeg, whatever the compression scheme;
  • The compression scheme seems to be applied at first glance (varying size for output tiff file);
  • Some software (e.g. Xnview) seem to decode incorrectly the files (black square inserted corresponding to the sampling block, then of course lines get shifted) anyway, so they should not be trusted.

I see the same broken output (green artefacts) as before with

ffmpeg -i yuv410p.avi out.tif

Changed 5 years ago by kurosu

Tentative fix

comment:10 follow-up: Changed 5 years ago by kurosu

Err, yes, I had a patch I thought fixed nothing applied locally, but it seems to actually fix things for me. Could you test the patch I've attached to this bug report?

comment:11 in reply to: ↑ 10 ; follow-up: Changed 5 years ago by ami_stuff

Replying to kurosu:

Err, yes, I had a patch I thought fixed nothing applied locally, but it seems to actually fix things for me. Could you test the patch I've attached to this bug report?

Applied, but effect is the same - green artefact when viewing output tiff with IrfanView?.

comment:12 in reply to: ↑ 11 ; follow-up: Changed 5 years ago by kurosu

  • Keywords tiffenc added

Replying to ami_stuff:

Applied, but effect is the same - green artefact when viewing output tiff with IrfanView?.

Don't test with IrfanView?, or Gimp, or XnView?. I've just tested, these ones have broken output, but libtiff (eg tiff2rgba) and ffmpeg (with the patch applied) do not. I suspect most softwares do not abide by the rule set in libtiff, encode extended image data if need be.

Second, are the softwares you use able to encode YUV with chroma subsampling? And is their output decodable by libtiff? By themselves? Are they able to decode libtiff output?

If you do have a file not produced by ffmpeg but that decodes well in both libtiff and some software, that would be interesting.

Last edited 5 years ago by kurosu (previous) (diff)

comment:13 follow-up: Changed 5 years ago by ami_stuff

Ok, so I downloaded some old ffmpeg autobuild from august and tiff2rgba decodes generated tiff file just fine
(I get no error messages and output looks correctly)

TIFF file generated with your patch decodes ok as well with tiff2rgba.

The only problem happens with other software like ImageMagick? and the rest which, as I understand, you belive is at fauly in some way. If this is true then this ticket can be closed.

Second, are the softwares you use able to encode YUV with chroma subsampling? And is their output decodable by libtiff? By themselves? Are they able to decode libtiff output?

I don't know any software (other than ffmpeg) which can generate YUV TIFFs.

C:\>ffmpeg -i yuv410p.avi out.tif
ffmpeg version N-65738-g3c197442 Copyright (c) 2000-2014 the FFmpeg developers
  built on Aug 18 2014 22:02:08 with gcc 4.8.3 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av
isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --
enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-lib
modplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrw
b --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinge
r --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --en
able-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis
 --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-
libx265 --enable-libxavs --enable-libxvid --enable-decklink --enable-zlib
  libavutil      54.  5.100 / 54.  5.100
  libavcodec     56.  0.101 / 56.  0.101
  libavformat    56.  1.100 / 56.  1.100
  libavdevice    56.  0.100 / 56.  0.100
  libavfilter     5.  0.100 /  5.  0.100
  libswscale      3.  0.100 /  3.  0.100
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  0.100 / 53.  0.100
Input #0, avi, from 'yuv410p.avi':
  Metadata:
    encoder         : Lavf54.3.100
  Duration: 00:00:00.04, start: 0.000000, bitrate: 9856 kb/s
    Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv410p, 323x240, 25 fps, 25 t
br, 25 tbn, 25 tbc
Output #0, image2, to 'out.tif':
  Metadata:
    encoder         : Lavf56.1.100
    Stream #0:0: Video: tiff, yuv410p, 323x240, q=2-31, 200 kb/s, 25 fps, 25 tbn
, 25 tbc
    Metadata:
      encoder         : Lavc56.0.101 tiff
Stream mapping:
  Stream #0:0 -> #0:0 (ffv1 (native) -> tiff (native))
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
video:86kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing ov
erhead: unknown
C:\>tiff2rgba out.tif out2.tif
Last edited 5 years ago by ami_stuff (previous) (diff)

Changed 5 years ago by kurosu

YUV 4:1:0 generated with rgb2ycbcr

comment:15 in reply to: ↑ 13 ; follow-up: Changed 5 years ago by kurosu

Replying to ami_stuff:

The only problem happens with other software like ImageMagick? and the rest which, as I understand, you belive is at fauly in some way. If this is true then this ticket can be closed.

Yes, try the attached yuv.tiff generated by:
ffmpeg -i lena.pnm -s 323x240 lena.tiff
rgb2ycbcr -c none -h 4 -v 4 lena.tiff yuv.tiff

Gimp displays green garbages at the bottom of stripes I suspect to be a multiple of 4 in height. XnView? shows staircases. ffmpeg decoded output is fine.

I don't know any software (other than ffmpeg) which can generate YUV TIFFs.

Please test them with the attached file or one generated following your preferences.

Regarding the yuv data, the real issue is rather the sampling. See the h/v parameters to rgbycbcr:
http://www.remotesensing.org/libtiff/man/rgb2ycbcr.1.html

comment:16 follow-up: Changed 5 years ago by kurosu

Note, ffmpeg encodes fine only with the attached patch.

I'll edit this patch before sending it in case you agree it's at least a partial fix.

comment:17 in reply to: ↑ 16 Changed 5 years ago by ami_stuff

Replying to kurosu:

Note, ffmpeg encodes fine only with the attached patch.

I'll edit this patch before sending it in case you agree it's at least a partial fix.

Please go ahead.

comment:18 in reply to: ↑ 15 ; follow-up: Changed 5 years ago by ami_stuff

Replying to kurosu:

Yes, try the attached yuv.tiff generated by:
ffmpeg -i lena.pnm -s 323x240 lena.tiff
rgb2ycbcr -c none -h 4 -v 4 lena.tiff yuv.tiff

Gimp displays green garbages at the bottom of stripes I suspect to be a multiple of 4 in height. XnView? shows staircases. ffmpeg decoded output is fine.

I don't know any software (other than ffmpeg) which can generate YUV TIFFs.

Please test them with the attached file or one generated following your preferences.

I can confirm that there are visible the same artefacts like with tiff file generated by ffmpeg.

comment:19 in reply to: ↑ 18 ; follow-up: Changed 5 years ago by kurosu

Replying to ami_stuff:

I can confirm that there are visible the same artefacts like with tiff file generated by ffmpeg.

So if I'm understanding it well, once the patch is applied, we will be able to summarize the new situation as "non-ffmpeg decoder bugs, the spec-compliant ones (including ffmpeg) are fine".

comment:20 in reply to: ↑ 19 Changed 5 years ago by ami_stuff

Replying to kurosu:

Replying to ami_stuff:

I can confirm that there are visible the same artefacts like with tiff file generated by ffmpeg.

So if I'm understanding it well, once the patch is applied, we will be able to summarize the new situation as "non-ffmpeg decoder bugs, the spec-compliant ones (including ffmpeg) are fine".

Yes, I think so.

comment:21 Changed 5 years ago by michael

  • Resolution set to fixed
  • Status changed from open to closed

(if i understand correctly) Fixed by Christophe Gisquet in 0e8bfd3c934768f9812dd20d71fa4709de54186d

Note: See TracTickets for help on using tickets.