Opened 18 months ago

Closed 18 months ago

Last modified 18 months ago

#5598 closed defect (fixed)

image2pipe disruption and OOM crash on BMP sequences with certain pixel values

Reported by: CoJaBo Owned by:
Priority: important Component: avcodec
Version: git-master Keywords: bmp regression
Cc: michael Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Bitmap images read through -f image2pipe disrupt ffmpeg if they contain values equal to "BM" at certain offsets; I suspect that ffmpeg is interpreting these as headers for some reason, even though they occur in the middle of image data.

The shortest sequence I can get to exhibit this issue is 3 frames. Run the following commands using the attached files and observe the number of output frames:

With a stream of 3 blank images, everything is as expected, with 3 frames output-

cat blank.bmp blank.bmp blank.bmp | ffmpeg -f image2pipe -i - junk.mkv

Changing 2 specific pixels near the top of the middle image results in ffmpeg seeing only 2 frames-

cat blank.bmp error.bmp blank.bmp | ffmpeg -f image2pipe -i - junk.mkv

If such a frame occurs in the middle of a large input stream, upon hitting it ffmpeg will stop producing output and attempt to read the remainder of the input stream into RAM until it runs out of memory and dies.

Attachments (2)

image2pipe-oom-crash.tar.gz (12.0 KB) - added by CoJaBo 18 months ago.
Sample bitmaps exhibiting the crash
image2pipe-oom-crash-2.tar.gz (12.1 KB) - added by CoJaBo 18 months ago.
Another set of bitmaps exhibiting the crash

Download all attachments as: .zip

Change History (8)

Changed 18 months ago by CoJaBo

Sample bitmaps exhibiting the crash

comment:1 in reply to: ↑ description ; follow-up: Changed 18 months ago by cehoyos

  • Component changed from undetermined to avcodec
  • Keywords bmp regression added

Replying to CoJaBo:

cat blank.bmp error.bmp blank.bmp | ffmpeg -f image2pipe -i - junk.mkv

Is -f image2pipe necessary to reproduce? I ask because generally forcing an input format can make a ticket less important.

If such a frame occurs in the middle of a large input stream, upon hitting it ffmpeg will stop producing output and attempt to read the remainder of the input stream into RAM until it runs out of memory and dies.

How can I reproduce this?

So far you have reported that input exists for which the bmp parser is unable to produce correct output. This is a regression for the given input but it may be "wontfix" (because parsing bmp can be difficult).

comment:2 in reply to: ↑ 1 Changed 18 months ago by CoJaBo

Replying to cehoyos:

Is -f image2pipe necessary to reproduce? I ask because generally forcing an input format can make a ticket less important.

It actually is not required; testing without it, I see that ffmpeg auto-detects the format as bmp_pipe instead of image2pipe, but this still exhibits the described bug.

If such a frame occurs in the middle of a large input stream, upon hitting it ffmpeg will stop producing output and attempt to read the remainder of the input stream into RAM until it runs out of memory and dies.

How can I reproduce this?

Just repeat the blank frame until the input stream won't fit in RAM; here's a one-liner to do this (increase 500 as needed):

(cat blank.bmp error.bmp; for i in `seq 1 500`; do cat blank.bmp; done;) | ffmpeg -i - junk.mkv

For comparison, it works with only the blank frame, generating the expected 502 frames of output with no errors:

(cat blank.bmp blank.bmp; for i in `seq 1 500`; do cat blank.bmp; done;) | ffmpeg -i - junk.mkv

So far you have reported that input exists for which the bmp parser is unable to produce correct output. This is a regression for the given input but it may be "wontfix" (because parsing bmp can be difficult).

Should it be that difficult though? The size of the file including header is the 4 bytes at offset 0x02; once the format has been identified as BMP, ffmpeg should read that many bytes before expecting either another header or EOF. Instead, it seems to be continuously scanning for anything that appears to be another header; that's only a 2-byte sequence (BM), so it will inevitably happen.

comment:3 Changed 18 months ago by michael

  • Reproduced by developer set
  • Resolution set to fixed
  • Status changed from new to closed

Changed 18 months ago by CoJaBo

Another set of bitmaps exhibiting the crash

comment:4 Changed 18 months ago by CoJaBo

  • Resolution fixed deleted
  • Status changed from closed to reopened

The fix above doesn't seem to have completely resolved the issue. Attached is another set of files that still exhibits the bug.

The cause is more or less the same, ffmpeg is still looking for a BMP header located randomly in the image data-

00000000  42 4d 36 ec 5e 00 00 00  00 00 36 00 00 00 28 00  |BM6.^.....6...(.|
00000010  00 00 80 07 00 00 38 04  00 00 01 00 18 00 00 00  |......8.........|
00000020  00 00 00 ec 5e 00 00 00  00 00 00 00 00 00 00 00  |....^...........|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00137cb0  00 00 00 00 00 42 4d 51  39 44 47 2f 37 3d 16 1d  |.....BMQ9DG/7=..|
00137cc0  24 02 09 0e 00 00 00 00  00 00 00 00 00 00 00 00  |$...............|
00137cd0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
005eec36

Any fix for this is going to need to actually correct that behavior and look for headers only after the end of the preceding file; I'm not sure from the diff if the previous patch attempted this or simply made the header check more strict. It goes without saying that BMP headers are loose enough to occur by chance every few minutes in HD video, and it undoubtedly gets worse at 4K.

To reproduce:

(for i in `seq 1 3982`; do cat blank.bmp; done; cat error2.bmp;) | ffmpeg -i - -f null -

An error is generated this time, and the output is 3982 frames instead of the expected 3983.

And here is the variant that causes an OOM crash (again, you'll need to increase 500 if you have more than about 4GB RAM):

(for i in `seq 1 3982`; do cat blank.bmp; done; cat error2.bmp; for i in `seq 1 500`; do cat blank.bmp; done;) | ffmpeg -i - -f null -

In both cases, I'm using -f null to speed things up (since we're reading nearly 25GB of data); using ffmpeg -i - junk.mkv as before will fail exactly the same way.

As an aside, is there a way to have ffmpeg (or another OSS tool) generate a BMP stream? This would be much easier to exhaustively test if there were, as simply transcoding any 1080p or 4K video and feeding it to ffmpeg would replicate this very quickly.

comment:5 Changed 18 months ago by michael

  • Resolution set to fixed
  • Status changed from reopened to closed

comment:6 Changed 18 months ago by michael

  • Cc michael added
Note: See TracTickets for help on using tickets.