A possible divide by zero bug

[YiyuanGUO]

Summary of the bug:
In libavfilter/vf_palettegen.c, the function get_avg_color has a potential divide by zero problem:

    int i;
    const int n = box->len;
    uint64_t r = 0, g = 0, b = 0, div = 0;

    for (i = 0; i < n; i++) {
        const struct color_ref *ref = refs[box->start + i];
        r += (ref->color >> 16 & 0xff) * ref->count;
        g += (ref->color >>  8 & 0xff) * ref->count;
        b += (ref->color       & 0xff) * ref->count;
        div += ref->count;
    }

    r = r / div;
    g = g / div;
    b = b / div;

If box->len equals to 0, then div remains 0 after the loop and triggers divide by zero problems.

This may happen through the following call sequences in vf_palettegen.c (if ctx->priv->nb_refs equals to 0):

​filter_frame -> ​get_palette_frame -> ​get_avg_color

Notice that the request_frame function has explicitly checked that the nb_refs field is nonzero before calling the function get_palette_frame to avoid such problems (link to the ​code):

    if (r == AVERROR_EOF && !s->palette_pushed && s->nb_refs && s->stats_mode != STATS_MODE_SINGLE_FRAMES) {
        r = ff_filter_frame(outlink, get_palette_frame(ctx));
    ......

Therefore, I think we need similar checks in the above mentioned call sequence.

This is a potential bug found by static analysis, and currently I don't have a POC. Please take a look and check if a fix is needed, thanks!

[Balling]

Fix is here ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210515055435.69717-1-yguoaz@gmail.com/

[Elon Musk]

Fixed in 2201c91fbf672447aa2063bd752d821466b8f11b