Opened 5 years ago

Last modified 5 years ago

#5223 new defect

avcodec_find_best_pix_fmt_of_list depends on the order?

Reported by: MattE Owned by:
Priority: normal Component: avutil
Version: Keywords:
Cc: devernay Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no


avcodec_find_best_pix_fmt_of_list seems to be returning different values depending on the order of the input.

Following is a example using python/cython, but it should be easy to understand:

print <int>avcodec_find_best_pix_fmt_of_list(fmts2, AV_PIX_FMT_YUV420P, 0, &loss)
print <int>avcodec_find_best_pix_fmt_of_list(fmts3, AV_PIX_FMT_YUV420P, 0, &loss)

Which prints:


I.e. if rgb24 is first that is returned, otherwise gray is preferred. If this is somewhat intentional, then it should be documented as such as as well as mentioned that the highest quality pixel formats should be listed first.

$ ffmpeg
ffmpeg version N-76278-gd897d4c Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 5.2.0 (GCC)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enable-zlib
  libavutil      55.  4.100 / 55.  4.100
  libavcodec     57. 10.100 / 57. 10.100
  libavformat    57. 11.100 / 57. 11.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6. 14.100 /  6. 14.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.100 /  2.  0.100
  libpostproc    54.  0.100 / 54.  0.100

Change History (5)

comment:1 follow-up: Changed 5 years ago by devernay

The "considered" loss mask is not the same for all pixel formats (the loss of first format is used a a mask for the second, etc) so that if the first pixel format loses resolution, resolution loss is not considered for other formats.

A workaround is to always pass NULL as the last parameter. This really returns the best pixel format.

A real fix is the following code, which uses the same "considered loss" mask for all formats in the list:

enum AVPixelFormat avcodec_find_best_pix_fmt_of_list(const enum AVPixelFormat *pix_fmt_list,
                                            enum AVPixelFormat src_pix_fmt,
                                            int has_alpha, int *loss_ptr){
    int i;

    enum AVPixelFormat best = AV_PIX_FMT_NONE;

    for(i=0; pix_fmt_list[i] != AV_PIX_FMT_NONE; i++) {
        int loss = loss_ptr ? *loss_ptr : 0;
        best = avcodec_find_best_pix_fmt_of_2(best, pix_fmt_list[i], src_pix_fmt, has_alpha, &loss);
    if (loss_ptr) {
        *loss_ptr = av_get_pix_fmt_loss(best,  src_pix_fmt, has_alpha);
    return best;

comment:2 in reply to: ↑ 1 Changed 5 years ago by cehoyos

  • Cc devernay added

Replying to devernay:

A real fix is the following code, which uses the same "considered loss" mask for all formats in the list:

Please send your patch made with git format-patch to the development mailing list, patches on this bug tracker are ignored.

comment:3 Changed 5 years ago by devernay

patch submitted to the mailing list

comment:4 Changed 5 years ago by cehoyos

No, unfortunately not;-(

comment:5 Changed 5 years ago by devernay

it's being moderated (I'm not an ffmpeg-devel list member)

Note: See TracTickets for help on using tickets.