Opened 6 years ago
Last modified 6 years ago
#6763 new defect
swscale: Out-of-bounds memory accesses
|Reported by:||Gramner||Owned by:|
|Blocking:||Reproduced by developer:||no|
|Analyzed by developer:||no|
Many assembly functions in swscale will read past the end of their input buffers which causes segfaults and/or bus errors if the buffer happens to be located near the end of a memory page and the next page is invalid.
Aligning input buffers isn't even enough for formats like RGB24 (and requiring alignment would be a bad idea anyway since it wouldn't work with memory-mapped input files for example).
Using swscale with x264 CLI seems to be a fairly consistent way to trigger such out-of-bounds crashes. This command line for example will cause segfaults in ff_rgb24ToY_avx():
./x264 -o /dev/null --input-csp rgb --input-res 512x512 <any_input_file>
If asm is disabled in swscale the problem goes away.
Change History (2)
comment:1 by , 6 years ago
|Priority:||normal → important|
comment:2 by , 6 years ago
|Version:||unspecified → git-master|
Not sure which disassembly you're interested in, but the source shows the entire row loop done using 128-bit loads with no special handling for the tail: http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libswscale/x86/input.asm;h=af9afcaa53a74f51eaa1257bafd1052524046074;hb=HEAD#l243
With width=512 as an example, on the last loop iteration "movu m4, [srcq+12]" reads 16 bytes from offset 1524 which results in an overflow of 4 bytes.
The input buffer is the exact size of the input data with zero padding as no such requirement is documented.
(Version information, backtrace, disassembly and register dump missing.)
Could you confirm that your input buffers are sufficiently padded?