Opened 3 years ago

Closed 3 years ago

Last modified 7 months ago

#7790 closed enhancement (invalid)

Endianness detection breaks when LTO is enabled through means other than the ffmpeg --enable-lto option

Reported by: RedDwarf82 Owned by:
Priority: normal Component: build system
Version: unspecified Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

ffmpeg's configure wrongly thinks my toolchain uses little-endian because of LTO.

$ cat test.c
unsigned int endian = 'B' << 24 | 'I' << 16 | 'G' << 8 | 'E';
$ gcc -c -o test.o test.c 
$ od -t x1 test.o | grep '42 *49 *47 *45'
$ gcc -c -o test.o test.c -fno-lto
$ od -t x1 test.o | grep '42 *49 *47 *45'
0000160 42 49 47 45 00 47 43 43 3a 20 28 42 75 69 6c 64

Change History (9)

comment:1 by RedDwarf82, 3 years ago

comment:2 by Hendrik, 3 years ago

If you intend to build ffmpeg with LTO enabled, you should do so by using the --enable-lto switch, which will only enable LTO after such tests are being performed, and not enable it behind FFmpegs back.

comment:3 by Carl Eugen Hoyos, 3 years ago

Version: 4.1unspecified

I cannot reproduce: --enable-lto does not break big-endian detection on different systems with different compilers I tested.
Please test current FFmpeg git head (nothing else is supported here) and tell us about your configure line, compiler and operating system (gcc -v, uname -a).

comment:4 by RedDwarf82, 3 years ago

It's a buildroot gcc build. I can see "-flto" was added to BR2_TARGET_OPTIMIZATION, which is why it's the default.

I can workaround my specific case. And I don't expect lots of people (but probably more than 1) to have gcc with LTO enabled by default. So feel free to close the issue if you want to.

But at the same time it seems to me that depending on people not enabling it "behind FFmpegs back" is error-prone.
I would say at the very least the check in https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/c2a221c5ae5017e11654b9688ac97e9f5d3570b2:/configure#l5596 should also check if it can find "45 *47 *49 *42". If it can't find either configure should fail.

comment:5 by RedDwarf82, 3 years ago

Summary: Endianness detection breaks with LTOEndianness detection breaks when LTO is enabled through means other than the ffmpeg --enable-lto option

comment:6 by Carl Eugen Hoyos, 3 years ago

Resolution: needs_more_info
Status: newclosed

comment:7 by RedDwarf82, 3 years ago

Resolution: needs_more_info
Status: closedreopened
Type: defectenhancement

Reopening since I really don't think this is a "needs_more_info" case. heleppkes has clearly described the issue. Also because doesn't even require the compiler to use LTO by default, so this may be a more common issue than I suggested before. Probably it's not more reported only because the more common desktop architecture happens to be little endian.

But here you have the extra information requested:

$ git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg
Cloning into 'ffmpeg'...
remote: Counting objects: 563150, done.
remote: Compressing objects: 100% (118600/118600), done.
remote: Total 563150 (delta 450571), reused 554797 (delta 443403)
Receiving objects: 100% (563150/563150), 133.44 MiB | 6.03 MiB/s, done.
Resolving deltas: 100% (450571/450571), done.
$ cd ffmpeg/
$ git rev-parse HEAD
0321370601833f4ae47e8e11c44570ea4bd382a4
$ mkdir build_normal build_lto
$ cd build_normal/
$ ../configure --enable-cross-compile --cross-prefix=mips-linux- --target-os=linux --arch=mips --cpu=74kc --disable-all
install prefix            /usr/local
source path               /opt/ffmpeg
C compiler                mips-linux-gcc
C library                 uclibc
host C compiler           gcc
host C library            glibc
ARCH                      mips (74kc)
big-endian                yes
runtime cpu detection     yes
MIPS FPU enabled          no
MIPS DSP R1 enabled       yes
MIPS DSP R2 enabled       yes
MIPS MSA enabled          no
LOONGSON MMI enabled      no
debug symbols             yes
strip symbols             yes
optimize for size         no
optimizations             yes
static                    yes
shared                    no
postprocessing support    no
network support           no
threading support         pthreads
safe bitstream reader     yes
texi2html enabled         no
perl enabled              yes
pod2man enabled           yes
makeinfo enabled          no
makeinfo supports HTML    no

External libraries:
alsa			  bzlib			    zlib

External libraries providing hardware acceleration:

Libraries:
avutil

Programs:

Enabled decoders:

Enabled encoders:

Enabled hwaccels:

Enabled parsers:

Enabled demuxers:

Enabled muxers:

Enabled protocols:

Enabled filters:

Enabled bsfs:

Enabled indevs:

Enabled outdevs:

License: LGPL version 2.1 or later

WARNING: mips-linux-pkg-config not found, library detection may fail.
$ cd ../build_lto
$ CFLAGS='-flto' ../configure --enable-cross-compile --cross-prefix=mips-linux- --target-os=linux --arch=mips --cpu=74kc --disable-all --enable-lto
install prefix            /usr/local
source path               /opt/ffmpeg
C compiler                mips-linux-gcc
C library                 uclibc
host C compiler           gcc
host C library            glibc
ARCH                      mips (74kc)
big-endian                no
runtime cpu detection     yes
MIPS FPU enabled          no
MIPS DSP R1 enabled       yes
MIPS DSP R2 enabled       yes
MIPS MSA enabled          no
LOONGSON MMI enabled      yes
debug symbols             yes
strip symbols             yes
optimize for size         no
optimizations             yes
static                    yes
shared                    no
postprocessing support    no
network support           no
threading support         pthreads
safe bitstream reader     yes
texi2html enabled         no
perl enabled              yes
pod2man enabled           yes
makeinfo enabled          no
makeinfo supports HTML    no

External libraries:
alsa			  bzlib			    zlib

External libraries providing hardware acceleration:

Libraries:
avutil

Programs:

Enabled decoders:

Enabled encoders:

Enabled hwaccels:

Enabled parsers:

Enabled demuxers:

Enabled muxers:

Enabled protocols:

Enabled filters:

Enabled bsfs:

Enabled indevs:

Enabled outdevs:

License: LGPL version 2.1 or later

WARNING: mips-linux-pkg-config not found, library detection may fail.
$ uname -a
Linux 7d04a811e585 4.20.13-200.fc29.x86_64 #1 SMP Wed Feb 27 19:42:55 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$ mips-linux-gcc -v
Using built-in specs.
COLLECT_GCC=/opt/buildroot_wdr3600/output/host/bin/mips-linux-gcc.br_real
COLLECT_LTO_WRAPPER=/opt/buildroot_wdr3600/output/host/libexec/gcc/mips-buildroot-linux-uclibc/7.4.0/lto-wrapper
Target: mips-buildroot-linux-uclibc
Configured with: ./configure --prefix=/opt/buildroot_wdr3600/output/host --sysconfdir=/opt/buildroot_wdr3600/output/host/etc --enable-static --disable-libstdcxx-dual-abi --disable-symvers --target=mips-buildroot-linux-uclibc --with-sysroot=/opt/buildroot_wdr3600/output/host/mips-buildroot-linux-uclibc/sysroot --enable-__cxa_atexit --with-gnu-ld --disable-libssp --disable-multilib --disable-decimal-float --with-gmp=/opt/buildroot_wdr3600/output/host --with-mpc=/opt/buildroot_wdr3600/output/host --with-mpfr=/opt/buildroot_wdr3600/output/host --with-pkgversion='Buildroot 2019.02-00010-g07502cdd25-dirty' --with-bugurl=http://bugs.buildroot.net/ --disable-libquadmath --disable-libsanitizer --enable-tls --enable-plugins --enable-lto --disable-libmudflap --enable-threads --without-isl --without-cloog --with-float=soft --with-arch=mips32r2 --with-abi=32 --with-nan=legacy --enable-languages=c,c++ --with-build-time-tools=/opt/buildroot_wdr3600/output/host/mips-buildroot-linux-uclibc/bin --enable-shared --disable-libgomp
Thread model: posix
gcc version 7.4.0 (Buildroot 2019.02-00010-g07502cdd25-dirty) 
$

I really don't think it's OK for ffmpeg to just say "it's the user's fault for using CFLAGS='-flto'" ("enable LTO behind FFmpegs back"). At the very least the configure scripts needs to notice the situation and report failure.

Notice the "LOONGSON MMI" detection is also broken when using -flto.

in reply to:  7 comment:8 by Carl Eugen Hoyos, 3 years ago

Resolution: invalid
Status: reopenedclosed

Replying to RedDwarf82:

$ CFLAGS='-flto' ../configure --enable-cross-compile --cross-prefix=mips-linux- --target-os=linux --arch=mips --cpu=74kc --disable-all --enable-lto

Thank you for the explanation!
CFLAGS (and --extra-cflags) can make the configure line invalid, in general this cannot be detected so it makes little sense to detect it in some exceptional cases.

comment:9 by RedDwarf, 7 months ago

FWIW nowadays LTO can have different options. Just from gcc/clang:

-flto
-flto=auto
-flto=thin
-flto=4

And by not letting the user specify LTO in the flags he is losing access to them.

Note: See TracTickets for help on using tickets.