checkasm float_dsp.vector_dmul sometimes fails on i386
|Reported by:||James Cowgill||Owned by:|
|Blocking:||Reproduced by developer:||no|
|Analyzed by developer:||no|
Sometimes (about 1 in 10 times), the float_dsp.vector_dmul checkasm test fails on i386.
$ tests/checkasm/checkasm --test=float_dsp 2642491962 checkasm: using random seed 2642491962 SSE: - float_dsp.vector_fmul [OK] - float_dsp.vector_fmac [OK] - float_dsp.butterflies_float [OK] - float_dsp.scalarproduct_float [OK] 93: -53.395181798898 - -53.395181798898 = -7.1054273576e-15 SSE2: vector_dmul_scalar_sse2 (float_dsp.c:171) - float_dsp.vector_dmul [FAILED] - float_dsp.vector_dmac [OK]
The failures are reprodicible given the same random seed. Note you probably need a toolchain which uses the x87 FPU to reproduce this (eg Fedora's gcc will not work).
I think this happens because the SSE routines compute the multiplication to double precision, but the reference C routines compute the multiplcation to 80-bit precision. Sometimes the results are different.
I noticed this patch which I think is an attempt to solve this. While it does reduce the probability of failure, I can still get the test to fail when run > 1000 times.
Downstream bug: https://bugs.debian.org/882075