Opened 2 years ago

Closed 2 years ago

#9945 closed defect (fixed)

undefined behavior at libavcodec/jpeg2000dwt.c:84:18

Reported by: Johannes Kauffmann Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: j2k ubsan
Cc: Johannes Kauffmann Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

UBSan complains about undefined behaviour when running make check

How to reproduce:

mkdir build && cd build
../configure --toolchain=gcc-usan --enable-debug=2 --disable-stripping --disable-doc
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 make -j8 check

... which prints:

Test j2k-dwt failed. Look at tests/data/fate/j2k-dwt.err for details.
make: *** [/home/user/ffmpeg/tests/Makefile:307: fate-j2k-dwt] Error 1
make: *** Waiting for unfinished jobs....

j2k-dwt.err contains:

src/libavcodec/jpeg2000dwt.c:84:18: runtime error: left shift of negative value -66
    #0 0x55863fe454bd in ff_dwt_encode.cold (/home/user/ffmpeg/build/libavcodec/tests/jpeg2000dwt+0x144bd)
    #1 0x55863fe5a046 in test_dwt src/libavcodec/tests/jpeg2000dwt.c:44
    #2 0x55863fe465f0 in main src/libavcodec/tests/jpeg2000dwt.c:134
    #3 0x7fd8ddf2fd8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #4 0x7fd8ddf2fe3f in __libc_start_main_impl ../csu/libc-start.c:392
    #5 0x55863fe48204 in _start (/home/user/ffmpeg/build/libavcodec/tests/jpeg2000dwt+0x17204)

Tested with:
ffmpeg git dd2ea014ef273157fe9a0e928e77841fbbee9b2f
GCC 11.2.0
Ubuntu 22.04.

Change History (4)

comment:1 by Carl Eugen Hoyos, 2 years ago

Keywords: j2k added; jpeg2000dwt removed

comment:2 by Carl Eugen Hoyos, 2 years ago

Keywords: ubsan added
$ make libavcodec/tests/jpeg2000dwt && libavcodec/tests/jpeg2000dwt
5/3i, decomp:15 border 151 170 140 183 milli-err2:        0
9/7i, decomp:15 border 151 170 140 183 milli-err2:      544
9/7f, decomp:15 border 151 170 140 183 err2:               0.000
5/3i, decomp:21 border 173 201  81 189 milli-err2:        0
9/7i, decomp:21 border 173 201  81 189 milli-err2:      592
9/7f, decomp:21 border 173 201  81 189 err2:               0.000
src/libavcodec/jpeg2000dwt.c:84:18: runtime error: left shift of negative value -66
5/3i, decomp:22 border 213 227  76 245 milli-err2:        0
9/7i, decomp:22 border 213 227  76 245 milli-err2:      533
9/7f, decomp:22 border 213 227  76 245 err2:               0.000
5/3i, decomp:13 border 134 157 184 203 milli-err2:        0
9/7i, decomp:13 border 134 157 184 203 milli-err2:      535
9/7f, decomp:13 border 134 157 184 203 err2:               0.000
5/3i, decomp: 1 border 204 237   6 106 milli-err2:        0
9/7i, decomp: 1 border 204 237   6 106 milli-err2:      219
9/7f, decomp: 1 border 204 237   6 106 err2:               0.000
5/3i, decomp:28 border  76 211  13 210 milli-err2:        0
9/7i, decomp:28 border  76 211  13 210 milli-err2:      791
9/7f, decomp:28 border  76 211  13 210 err2:               0.000
5/3i, decomp:21 border  76  99  43 123 milli-err2:        0
9/7i, decomp:21 border  76  99  43 123 milli-err2:      686
9/7f, decomp:21 border  76  99  43 123 err2:               0.000
5/3i, decomp:15 border 192 243 174 204 milli-err2:        0
9/7i, decomp:15 border 192 243 174 204 milli-err2:      476
9/7f, decomp:15 border 192 243 174 204 err2:               0.000
5/3i, decomp:21 border  17  68  93 204 milli-err2:        0
9/7i, decomp:21 border  17  68  93 204 milli-err2:      633
9/7f, decomp:21 border  17  68  93 204 err2:               0.000
5/3i, decomp:11 border 142 168  82 174 milli-err2:        0
9/7i, decomp:11 border 142 168  82 174 milli-err2:      696
9/7f, decomp:11 border 142 168  82 174 err2:               0.000
5/3i, decomp:23 border 142 209 171 235 milli-err2:        0
9/7i, decomp:23 border 142 209 171 235 milli-err2:      626
9/7f, decomp:23 border 142 209 171 235 err2:               0.000
5/3i, decomp:30 border  37 185  79 245 milli-err2:        0
9/7i, decomp:30 border  37 185  79 245 milli-err2:      953
9/7f, decomp:30 border  37 185  79 245 err2:               0.000
5/3i, decomp: 5 border 129 236  30 243 milli-err2:        0
9/7i, decomp: 5 border 129 236  30 243 milli-err2:      620
9/7f, decomp: 5 border 129 236  30 243 err2:               0.000
5/3i, decomp:10 border   5 160 146 247 milli-err2:        0
9/7i, decomp:10 border   5 160 146 247 milli-err2:      797
9/7f, decomp:10 border   5 160 146 247 err2:               0.000
5/3i, decomp: 5 border 104 162   6  47 milli-err2:        0
9/7i, decomp: 5 border 104 162   6  47 milli-err2:      603
9/7f, decomp: 5 border 104 162   6  47 err2:               0.000
5/3i, decomp:24 border  78 250 102 218 milli-err2:        0
9/7i, decomp:24 border  78 250 102 218 milli-err2:      836
9/7f, decomp:24 border  78 250 102 218 err2:               0.000
5/3i, decomp:28 border  86  98  56  79 milli-err2:        0
9/7i, decomp:28 border  86  98  56  79 milli-err2:      597
9/7f, decomp:28 border  86  98  56  79 err2:               0.000
5/3i, decomp: 6 border  95 238 197 214 milli-err2:        0
9/7i, decomp: 6 border  95 238 197 214 milli-err2:      478
9/7f, decomp: 6 border  95 238 197 214 err2:               0.000
5/3i, decomp:17 border  77 169  93 165 milli-err2:        0
9/7i, decomp:17 border  77 169  93 165 milli-err2:      616
9/7f, decomp:17 border  77 169  93 165 err2:               0.000
5/3i, decomp:22 border 178 187   7 119 milli-err2:        0
9/7i, decomp:22 border 178 187   7 119 milli-err2:      392
9/7f, decomp:22 border 178 187   7 119 err2:               0.000

comment:3 by Balling, 2 years ago

Last edited 2 years ago by Balling (previous) (diff)

comment:4 by Balling, 2 years ago

Resolution: fixed
Status: newclosed
Note: See TracTickets for help on using tickets.