iff ilbm ham8 with masking decodes incorrectly

[ami_stuff]

decodes ok with amiga programs and pmview

​http://www.pmview.com/

C:\>ffmpeg -i ham8_transparent.iff out.bmp
ffmpeg version N-37208-g01fcbdf Copyright (c) 2000-2012 the FFmpeg developers
  built on Jan 27 2012 18:34:52 with gcc 4.6.2
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-ru
ntime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libope
ncore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --en
able-libmp3lame --enable-libopenjpeg --enable-librtmp --enable-libschroedinger -
-enable-libspeex --enable-libtheora --enable-libvo-aacenc --enable-libvo-amrwben
c --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-
libxvid --enable-zlib
  libavutil      51. 34.101 / 51. 34.101
  libavcodec     53. 60.100 / 53. 60.100
  libavformat    53. 31.100 / 53. 31.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 60.100 /  2. 60.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  6.100 /  0.  6.100
  libpostproc    52.  0.100 / 52.  0.100
[IFF @ 02142320] Estimating duration from bitrate, this may be inaccurate
Input #0, IFF, from 'ham8_transparent.iff':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: iff_byterun1 (ILBM / 0x4D424C49), bgra, 170x129, SAR 44:
44 DAR 170:129, 90k tbr, 90k tbn, 90k tbc
[image2 @ 02143760] Frame rate very high for a muxer not effciciently supporting
 it.
Please consider specifiying a lower framerate, a different muxer or -vsync 2
[buffer @ 021519A0] w:170 h:129 pixfmt:bgra tb:1/1000000 sar:44/44 sws_param:
Output #0, image2, to 'out.bmp':
  Metadata:
    encoder         : Lavf53.31.100
    Stream #0:0: Video: bmp, bgra, 170x129 [SAR 44:44 DAR 170:129], q=2-31, 200
kb/s, 90k tbn, 90k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (iff_byterun1 -> bmp)
Press [q] to stop, [?] for help
frame=    1 fps=  0 q=0.0 Lsize=       0kB time=00:00:00.00 bitrate=   0.0kbits/
s
video:86kB audio:0kB global headers:0kB muxing overhead -100.000000%

[Carl Eugen Hoyos]

Could you attach png images that show how the originals are supposed to look like?

[ami_stuff]

Attached.

[ami_stuff]

this fixes the colors for ham8 with masking (stencil):

libavcodec/iff.c:

        if (s->masking == MASK_HAS_MASK) {
-            if (s->bpp >= 8) { 
+            if (s->bpp >= 8 && !s->ham) { 
                avctx->pix_fmt = PIX_FMT_RGB32;
                av_freep(&s->mask_buf);
                av_freep(&s->mask_palbuf);
                s->mask_buf = av_malloc((s->planesize * 32) + FF_INPUT_BUFFER_PADDING_SIZE);
                if (!s->mask_buf)
                    return AVERROR(ENOMEM);
                s->mask_palbuf = av_malloc((2 << s->bpp) * sizeof(uint32_t) + FF_INPUT_BUFFER_PADDING_SIZE);
                if (!s->mask_palbuf) {
                    av_freep(&s->mask_buf);
                    return AVERROR(ENOMEM);
                }
            }
            s->bpp++;
        } else if (s->masking != MASK_NONE && s->masking != MASK_HAS_TRANSPARENT_COLOR) {
            av_log(avctx, AV_LOG_ERROR, "Masking not supported\n");
            return AVERROR_PATCHWELCOME;

but the problem with graphic artifacts still remains.

libsdl_image decodes correctly attached images.

[Carl Eugen Hoyos]

Patch reviewed by Peter Ross and pushed.