ffmpeg returns blank frames on some avi files with an initial -ss

[Pat Gunn]

Summary of the bug:
How to reproduce:

% fmpeg -ss 10 -i /home/pgunn/caiman_data/example_movies/msCam13.avi -ss 10 -loglevel info -vf scale=752:480 -sws_flags bicubic -pix_fmt rgb24 /tmp/foo_noprelinear.avi
ffmpeg version (tested on 5.x and 6.x on various platforms

With that initial -ss all the frames of the output movie are 0 (for this input).

Without it (just with the latter -ss, the output looks fine.

There are no interesting warnings that the "info" level of logging outputs with our files (datafiles generated directly by microscopes).

You may find one of these datafiles here (no need to register or anything, just use wget):
​https://caiman.flatironinstitute.org/~neuro/caiman_downloadables/msCam13.avi

[mkver]

The first packet of the video in this file has a palette that gets skipped in case of an input seek.

More exactly, the palette gets read from extradata and attached to a packet that is read during avformat_find_stream_info(), yet this packet is discarded upon seek. Using stream side data would probably fix this, yet this is problematic because there is a second place where this demuxer may add a palette and stream side data is supposed to be used for stuff that does not change at all.

[Pat Gunn]

Unfortunately, we don't have the ability to rework how the file is saved; typically microscopes have embedded software that write these file formats; I am one of the maintainers of a piece of scientific software that takes such files as input and does automated analysis on it. ffmpeg ends up used by this through moviepy through pims.

It's frustrating that the avi format makes writing a good parser nontrivial. If always reading the first packet for metadata (like the palette) before doing a seek is sensible, that'd be great.