tiff LZW compression does not compress enough

[jsteinkamp]

Summary of the bug:tiff LZW compression might not compress enough
How to reproduce:
ffmpeg -y -i b.tif -pix_fmt rgb24 -vcodec tiff -compression_algo lzw test_24.tif
The file made in ffmepg is 59.2 MB, if I resave the file in photoshop it is 39.8 MB
ffmpeg version
N-78598-g98a0053
Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

[Carl Eugen Hoyos]

To make this a valid ticket please test current FFmpeg git head, provide the command line you tested together with the complete, uncut console output and point us to the sample created by Photoshop.

[jsteinkamp]

Hi I updated to most current version.
I included the original (smaller) 32 bit file to be changed to 24 bit LZW. I also included the photoshop saved smaller file.

recap:
I convert a 32 bit file 50MB to 24 bit, it becomes 60.7MB. I resave it in photoshop with LZW compression it reduces to 40.8M

please download example files from: ​http://jsteinkamp.com/ffmpeg_tif.zip
Thanks
Jennifer

ffmpeg version N-81872-gbe1d324 Copyright (c) 2000-2016 the FFmpeg developers

built with gcc 5.4.0 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-libebur128 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
libavutil 55. 32.100 / 55. 32.100
libavcodec 57. 60.101 / 57. 60.101
libavformat 57. 51.102 / 57. 51.102
libavdevice 57. 0.102 / 57. 0.102
libavfilter 6. 63.100 / 6. 63.100
libswscale 4. 1.100 / 4. 1.100
libswresample 2. 2.100 / 2. 2.100
libpostproc 54. 0.100 / 54. 0.100

[tiff_pipe @ 00000000004fa7c0] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, tiff_pipe, from 'b.tif':

Duration: N/A, bitrate: N/A

Stream #0:0: Video: tiff, rgba, 2160x11520 [SAR 1:1 DAR 3:16], 25 tbr, 25 tbn, 25 tbc

Output #0, image2, to 'test_24.tif':

Metadata:

encoder : Lavf57.51.102
Stream #0:0: Video: tiff, rgb24, 2160x11520 [SAR 1:1 DAR 3:16], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
Metadata:

encoder : Lavc57.60.101 tiff

Stream mapping:

Stream #0:0 -> #0:0 (tiff (native) -> tiff (native))

Press [q] to stop, ? for help
frame= 1 fps=0.0 q=0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 1 fps=0.6 q=-0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A speed=0.026x
video:60706kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

[jsteinkamp]

jsteinkamp Replying to jsteinkamp:
HI, I tested this bug again.
The LZW compression for TIFF is still larger than photoshop.
photoshop file 716 KB, ffmpeg file 884KB
The file was 32 bit 1024x768 included an alpha channel

ffmpeg version: N-91024-g293a6e8332

built with gcc 7.3.0 (GCC)

configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --e

nable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libblur
ay --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-
libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enab
le-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-li
bvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --en
able-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-
libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enabl
e-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enabl
e-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enab
le-dxva2 --enable-avisynth

libavutil 56. 18.100 / 56. 18.100
libavcodec 58. 19.101 / 58. 19.101
libavformat 58. 13.102 / 58. 13.102
libavdevice 58. 4.100 / 58. 4.100
libavfilter 7. 21.100 / 7. 21.100
libswscale 5. 2.100 / 5. 2.100
libswresample 3. 2.100 / 3. 2.100
libpostproc 55. 2.100 / 55. 2.100

Input #0, tiff_pipe, from 'hibiscus1.0001.tif':

Duration: N/A, bitrate: N/A

Stream #0:0: Video: tiff, rgba, 1024x768 [SAR 1:1 DAR 4:3], 25 tbr, 25 tbn,

25 tbc
Stream mapping:

Stream #0:0 -> #0:0 (tiff (native) -> tiff (native))

Press [q] to stop, ? for help
Output #0, image2, to 'hibiscus1.0001.tif':

Metadata:

encoder : Lavf58.13.102
Stream #0:0: Video: tiff, rgba, 1024x768 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s

, 25 fps, 25 tbn, 25 tbc

Metadata:

encoder : Lavc58.19.101 tiff

frame= 1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A speed=0.526x

video:883kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing o
verhead: unknown

Summary of the bug:tiff LZW compression might not compress enough
How to reproduce:
ffmpeg -y -i b.tif -pix_fmt rgb24 -vcodec tiff -compression_algo lzw test_24.tif
The file made in ffmepg is 59.2 MB, if I resave the file in photoshop it is 39.8 MB
ffmpeg version
N-78598-g98a0053
Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

[Carl Eugen Hoyos]

Replying to jsteinkamp:

The LZW compression for TIFF is still larger than photoshop.
photoshop file 716 KB

Where can I find this sample?

[jsteinkamp]

Hi I made you a photoshop LZW compressed image at:
​http://jsteinkamp.com/saved_photoshop.zip

[James]

tiffcp from the libtiff package compresses it even more, from ~716k to ~691k. In contrast the output generated by libavcodec is ~884k.

Our lzw encoder is apparently inefficient, so changing this to an enhancement request.

[Balling]

Replying to jamrial:

tiffcp from the libtiff package compresses it even more, from ~716k to ~691k. In contrast the output generated by libavcodec is ~884k.

Our lzw encoder is apparently inefficient, so changing this to an enhancement request.

I think it is a good idea to update this with Photoshop 2020 and latest tiffcp...

[Balling]

libavcodec is better now 784234 Bytes... But !!! It does not store the alpha channel and if you will save tiff in photoshop 21.1 without alpha channel it will be... wait for it... 605392 Bytes! Just wow. With alpha channel it will be the same (algorithm in Photoshop is the same, byte stream is the same, though some metadata is different). Maybe will try to check libtiff.