sws_init_context crashes for destination width of 4 or 2 when doing any operation with FAST_BILINEAR

[Danilo Gasques]

Summary: I'm trying to scale an RGB32 image down from 8x8 to 4x4 using SWS_FAST_BILINEAR and sws_init_context crashes with an arithmetic exception.

I'm using lastest source code from git (10/23/2015 16:49AM GMT) on Linux x64, gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3

Build options:
./configure --enabled-shared --enable-debug=3 --disable-stripping

mytest.cpp

#include <stdio.h>                                                               
                                                                                 
#include "libavutil/avutil.h"                                                    
#include "libavutil/log.h"                                                       
#include "libavutil/pixfmt.h"                                                    
#include "libswscale/swscale.h"                                                  
#include "libavcodec/avcodec.h"                                                  
#include <math.h>                                                                
                                                                                 
                                                                                 
int main(int argc, char *argv[])                                                 
{                                                                                
                                                                                 
    AVPicture input;                                                             
    avpicture_alloc(&input, AV_PIX_FMT_RGB32, 8, 8);                             
                                                                                 
    uint32_t* pixels = (uint32_t*)input.data[0];                                 
                                                                                 
    // create a bitmap                                                           
    int i = 0;                                                                   
    for (i=0; i < 8*8; ++i)                                                      
    {                                                                            
            *pixels = 0xFF0000FF;                                                
            ++pixels;                                                            
    }                                                                            
                                                                                 
                                                                                 
    AVPicture output;                                                            
    avpicture_alloc(&output, AV_PIX_FMT_RGB32, 4, 4);                          
                                                                                 
    struct SwsContext *ctx = sws_getContext(8,8, AV_PIX_FMT_RGB32,               
                                            4,4, AV_PIX_FMT_RGB32, 
                                            SWS_FAST_BILINEAR, NULL, NULL, NULL);
                                                                                 
    if (ctx != NULL)                                                             
    {                                                                            
        sws_scale(ctx, (const uint8_t * const*) &input.data[0], &input.linesize[0], 0, 4, &output.data[0], &output.linesize[0]);                                                                             
        sws_freeContext(ctx);                                                    
    }                                                                            
                                                                                 
    avpicture_free(&output);                                                     
    avpicture_free(&input);                                                      
                                                                                 
                                                                                 
    return 0;                                                                    
}         

Running it gives:

Floating point exception (core dumped)

Inspecting the core file with gdb:

GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.launchpad.net/gdb-linaro/>...
Reading symbols from /home/danilo/.tmp/23-10-2015/ffmpeg/mytest...(no debugging symbols found)...done.
[New LWP 13273]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fff001ed000
Core was generated by `./mytest'.
Program terminated with signal 8, Arithmetic exception.
#0  0x00007f3beae229e8 in sws_init_context (c=0x1b2b220, srcFilter=0x7ffcc31f9300, dstFilter=0x7ffcc31f9300) at libswscale/utils.c:1391
1391	            c->chrXInc = ((int64_t)(c->chrSrcW - 2) << 16) / (c->chrDstW - 2) - 20;
(gdb) p c->chrDstW
$1 = 2
(gdb) p c->dstW
$2 = 4
(gdb) p c->srcW
$3 = 8
(gdb) bt full
#0  0x00007f3beae229e8 in sws_init_context (c=0x1b2b220, srcFilter=0x7ffcc31f9300, dstFilter=0x7ffcc31f9300) at libswscale/utils.c:1391
        i = <optimized out>
        j = <optimized out>
        usesVFilter = 0
        usesHFilter = 0
        unscaled = 0
        dummyFilter = {lumH = 0x0, lumV = 0x0, chrH = 0x0, chrV = 0x0}
        srcW = 8
        srcH = 8
        dstW = 4
        dstH = 4
        dst_stride = <optimized out>
        flags = 1
        cpu_flags = 21467
        srcFormat = AV_PIX_FMT_BGRA
        dstFormat = AV_PIX_FMT_BGRA
        desc_src = 0x7f3b00000002
        desc_dst = 0x7f3bea81f9e0
        ret = 0
        tmpFmt = <optimized out>
#1  0x00007f3beae7205e in sws_getContext (srcW=<optimized out>, srcH=<optimized out>, srcFormat=<optimized out>, dstW=<optimized out>, dstH=<optimized out>, dstFormat=<optimized out>, flags=1, 
    srcFilter=0x0, dstFilter=0x0, param=0x0) at libswscale/utils.c:1867
        c = 0x1b2b220
#2  0x0000000000400870 in main ()

The problem seems to be related to destination size other than anything else. I could also reproduce it with an input width equals to the destination width. Moreover, sws_init_context also divides by zero if the destination width equals to 2.

libswscale/utils.c:1390
1390	            c->lumXInc = ((int64_t)(srcW       - 2) << 16) / (dstW       - 2) - 20;

[Carl Eugen Hoyos]

Crashes since 425c30ddae3e6f36b80edad192a6108bfa50277e

[Michael Niedermayer]

fixed in 1edf129cbc897447a289ca8b045853df5df1bab3

btw, avpicture_alloc() is deprecated