Opened 8 years ago

Last modified 8 years ago

#1853 open defect

libswscale writes past scanline end

Reported by: gjdfgh Owned by:
Priority: normal Component: swscale
Version: git-master Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no


libswscale sometimes writes to the memory past the end of each scanline, into the space between end of scanline and start of the next pixel, or in other words, it overwrites pixels between (dstw, y) and (0, y+1).

Often, this memory is just padding to satisfy alignment constraints on strides, but there are use cases where it's reasonable to expect that pixels outside the specified destination width should not be overwritten. For example, the destination image could be set up as cropped region of a larger image, and the regions of the larger images not covered by the destination region should not be overwritten.

I suspect this bug/caveat happens due to use of SIMD to work on multiple pixels at once. libswscale should just use an unaccelerated code path to handle trailing pixels.

This happens at least with PIX_FMT_BGRA.

Attachments (1)

libswscale-test.c (7.5 KB) - added by divVerent 8 years ago.

Download all attachments as: .zip

Change History (6)

comment:1 Changed 8 years ago by cehoyos

This may be a serious bug, but without a command line, complete uncut console output and valgrind and/or gdb output this report is unfortunately completely unusable.

comment:2 Changed 8 years ago by compn

a sample and/or easy way to reproduce this would be most helpful in getting this problem fixed.

a patch would be nice too...

comment:3 Changed 8 years ago by richardpl

  • Cc added
  • Reproduced by developer set
  • Status changed from new to open
  • Version changed from unspecified to git-master

comment:4 Changed 8 years ago by cehoyos

Trying to address some of the comments made on irc about this ticket:
After working on several thousand FFmpeg bug reports (not just 1500 - ? - on trac) and a few mails on ffmpeg-users, I feel very safe to assume that reports that I can reproduce have a significantly higher probability of being fixed than reports that I cannot reproduce / do not understand.
So typically, it makes a lot of sense to report the problem in a way that it can be easily understood by somebody who is neither an English native speaker nor the original author of the code.
Concerning the question why ffmpeg (the application) has to be used: Of course tickets can be opened for library usage only, but as was explained on irc, in 90% of all such cases the problem is either easily reproducible with ffmpeg (and it is much more likely that a developer will work on it because a test case is available without additional time-consuming effort) or a bug in the user application. In the remaining cases, trying to figure out why a problem is not reproducible with ffmpeg typically helps a lot understanding (and fixing) the issue.
That brings me to another point that may also be relevant in this case: It is not only not the duty of the reporter to analyze an issue, it is actually his only duty to provide a reproducible report with all information needed to reproduce the problem. It is of course ok to add an analysis, but the analysis should never replace the actual report, it is purely optional.
And I will of course continue to also comment on issues that I fail to understand - it is not a good idea to ignore tickets: I originally missed ticket #1587 and it is now an interesting looking report without a sample to reproduce;-(

Or in other words: Please believe me that my comments are made with the intention to help the reporter (both with the original problems and with future reports).

comment:5 Changed 8 years ago by divVerent

The problem here is that it obviously cannot be reproduced with the command line tools, as they do not contain a way to position the output of libswscale into part of an image (e.g. a cropped section).

I wrote a C program for trying out these things, though, and am attaching it.

Test like this:

$ gcc libswscale-test.c -o libswscale-test -Wall -Wextra -lswscale -lavutil -lm
[rpolzer@nb-04 tmp]$ ./libswscale-test yuv444p yuv444p | head -n 25
TEST: 59x39 -> 13x100+0+1 @ 14x102: swsFlags=0x40040 OK
TEST: 107x127 -> 27x3+13+14 @ 69x37: invalid target dimensions, skipped
[swscaler @ 0x12e14a0] Warning: data is not aligned! This can lead to a speedloss
TEST: 60x41 -> 41x17+14+0 @ 62x29: swsFlags=0x48400 OK
TEST: 31x96 -> 53x38+0+0 @ 53x83: swsFlags=0x48002 OK
TEST: 100x74 -> 23x22+5+11 @ 28x39: swsFlags=0x1 OK
TEST: 86x106 -> 38x2+10+27 @ 76x37: invalid target dimensions, skipped
TEST: 25x14 -> 85x20+6+47 @ 95x75: swsFlags=0x40004 OK
TEST: 32x40 -> 115x38+4+4 @ 125x67: swsFlags=0xc8004 OK
TEST: 58x18 -> 2x5+8+59 @ 25x67: invalid target dimensions, skipped
TEST: 32x49 -> 6x76+21+4 @ 57x86: invalid target dimensions, skipped
TEST: 55x19 -> 60x35+46+2 @ 123x37: swsFlags=0xc0200 OK
TEST: 50x39 -> 15x5+8+1 @ 49x24: invalid target dimensions, skipped
TEST: 23x64 -> 40x12+38+6 @ 97x99: swsFlags=0xc8400 OK
TEST: 44x9 -> 18x54+28+40 @ 54x105: swsFlags=0x88080 OK
TEST: 117x58 -> 55x117+19+9 @ 88x128: swsFlags=0x80040 OK
TEST: 111x37 -> 12x7+28+6 @ 60x51: invalid target dimensions, skipped
TEST: 122x115 -> 6x34+1+8 @ 17x67: invalid target dimensions, skipped
TEST: 87x17 -> 29x11+6+13 @ 55x28: swsFlags=0xc8040 OK
TEST: 40x115 -> 24x55+0+61 @ 24x119: swsFlags=0x8010 OK
TEST: 50x101 -> 14x67+32+20 @ 117x99: swsFlags=0x48080 OK
TEST: 24x56 -> 23x89+21+13 @ 66x104: swsFlags=0x200 (failed in comp=1 x=0 y=20 expected=28 got=27) output image is not solid! (failed in comp=0 x=44 y=13 expected=72 got=121) wrote beyond bounds! FAILED
TEST: 63x27 -> 31x46+3+32 @ 40x103: swsFlags=0x8100 (failed in comp=0 x=0 y=41 expected=143 got=144) output image is not solid! (failed in comp=0 x=34 y=32 expected=227 got=143) wrote beyond bounds! FAILED
TEST: 95x51 -> 14x9+5+20 @ 24x109: swsFlags=0x40008 OK
TEST: 15x127 -> 80x12+13+11 @ 127x45: swsFlags=0xc0040 OK
TEST: 101x59 -> 16x52+4+18 @ 106x75: swsFlags=0x88080 OK

What it does:

It creates two rectangle areas i1 and i2.
Also, i3 is created aliasing a cropped region of i1.

Now i1 and i2 are filled with different solid colors. i2 is then taken and scaled into i3, which is that cropped region of i1.

Then it checks that i3 is filled with a single solid color (this sometimes fails due to bad filtering, but SWS_ACCURATE_RND prevents that, so ignore it) and afterwards checks that i1, with the region of i3 excluded, is also still filled with a single solid color (this check fails whenever libswscale wrote outside the target bounds).

In the first failing test output, for example, i2 was 24x56, i1 was 66x104 and i3 was a 23x89 section with its top left corner at 21,13. The first check failed because the pixel at 20,0 had U component value 27, while the previous pixel had 28 (typical roundoff error - this is the issue that SWS_ACCURATE_RND prevents). The second check however failed too, because in the Y component, pixel 44,13 was written to! 44,13 is the pixel on the right of the top right pixel of the 23x89 rectangle having its top left corner at 21,13. This is the issue we are having here.

Changed 8 years ago by divVerent

Note: See TracTickets for help on using tickets.