parallel minimalistic build fails

[barsnick]

Summary of the bug:
A parallel (e.g. -j4) build of ffmpeg fails consistently, under certain configuration circumstances, due to (probably) race conditions by incorrect dependencies. A second "make" attempt succeeds.

How to reproduce (wrapped for readability):

$ ../configure --disable-doc --disable-everything --disable-network --disable-vdpau \
  --enable-protocol=file,pipe --enable-indev=lavfi --enable-muxer=null,spdif \
  --enable-demuxer=matroska  --enable-encoder=wrapped_avframe,truehd \
  --enable-decoder=rawvideo,pcm_f64le \
  --enable-filter=anoisesrc,anullsrc,nullsrc,testsrc,null,aresample \
&& make -j4 -k clean ffmpeg_g

Result:

[... various warnings]
AR      libavutil/libavutil.a
STRIP   libavcodec/x86/simple_idct.o
strip: 'libavcodec/x86/simple_idct.o': No such file
make: [/home/barsnick/Development/ffmpeg-stuff/ffmpeg/ffbuild/common.mak:83: libavcodec/x86/simple_idct.o] Error 1 (ignored)
AR      libavcodec/libavcodec.a
ar: libavcodec/ac3_parser.o: No such file or directory
make: *** [/home/barsnick/Development/ffmpeg-stuff/ffmpeg/ffbuild/library.mak:22: libavcodec/libavcodec.a] Error 1
[... various warnings]
make: Target 'ffmpeg_g' not remade because of errors.

On a different machine (x86), I get:

STRIP   libavutil/x86/lls.o
CC      fftools/ffmpeg_filter.o
STRIP   libavutil/x86/float_dsp.o
CC      fftools/ffmpeg_hw.o
src/fftools/ffmpeg_opt.c: In function ‘add_input_streams’:
src/fftools/ffmpeg_opt.c:798: warning: ‘codec’ is deprecated (declared at src/libavformat/avformat.h:878)
src/fftools/ffmpeg_opt.c:799: warning: ‘codec’ is deprecated (declared at src/libavformat/avformat.h:878)
src/fftools/ffmpeg_opt.c:800: warning: ‘codec’ is deprecated (declared at src/libavformat/avformat.h:878)
src/fftools/ffmpeg_opt.c:801: warning: ‘codec’ is deprecated (declared at src/libavformat/avformat.h:878)
src/fftools/ffmpeg_opt.c:802: warning: ‘codec’ is deprecated (declared at src/libavformat/avformat.h:878)
src/fftools/ffmpeg_opt.c:803: warning: ‘codec’ is deprecated (declared at src/libavformat/avformat.h:878)
src/fftools/ffmpeg_opt.c: In function ‘open_output_file’:
src/fftools/ffmpeg_opt.c:2399: warning: ‘codec’ is deprecated (declared at src/libavformat/avformat.h:878)
CC      fftools/cmdutils.o
src/fftools/cmdutils.c: In function ‘print_all_libs_info’:
src/fftools/cmdutils.c:1134: warning: ‘avresample_version’ is deprecated (declared at src/libavresample/avresample.h:148)
src/fftools/cmdutils.c:1134: warning: ‘avresample_configuration’ is deprecated (declared at src/libavresample/avresample.h:158)
src/fftools/ffmpeg_hw.c: In function ‘hw_device_setup_for_decode’:
src/fftools/ffmpeg_hw.c:303: warning: ‘err’ may be used uninitialized in this function
src/fftools/ffmpeg_hw.c:301: warning: ‘type’ may be used uninitialized in this function
src/fftools/cmdutils.c: In function ‘next_codec_for_id’:
src/fftools/cmdutils.c:1499: warning: ‘av_codec_next’ is deprecated (declared at src/libavcodec/avcodec.h:4102)
CC      fftools/ffmpeg.o
src/fftools/ffmpeg.c: In function ‘do_streamcopy’:
src/fftools/ffmpeg.c:2073: warning: ‘av_copy_packet_side_data’ is deprecated (declared at src/libavcodec/avcodec.h:4447)
src/fftools/ffmpeg.c: In function ‘init_output_stream’:
src/fftools/ffmpeg.c:3544: warning: ‘avcodec_copy_context’ is deprecated (declared at src/libavcodec/avcodec.h:4219)
src/fftools/ffmpeg.c:3544: warning: ‘codec’ is deprecated (declared at src/libavformat/avformat.h:878)
src/fftools/ffmpeg.c:3590: warning: ‘codec’ is deprecated (declared at src/libavformat/avformat.h:878)
src/fftools/ffmpeg.c: In function ‘check_keyboard_interaction’:
src/fftools/ffmpeg.c:3971: warning: ‘codec’ is deprecated (declared at src/libavformat/avformat.h:878)
src/fftools/ffmpeg.c:3994: warning: ‘codec’ is deprecated (declared at src/libavformat/avformat.h:878)
src/fftools/ffmpeg_opt.c: In function ‘opt_map’:
src/fftools/ffmpeg_opt.c:275: warning: ‘allow_unused’ may be used uninitialized in this function
AR      libavdevice/libavdevice.a
AR      libavfilter/libavfilter.a
AR      libavformat/libavformat.a
AR      libavcodec/libavcodec.a
AR      libswresample/libswresample.a
AR      libswscale/libswscale.a
AR      libavutil/libavutil.a
src/fftools/ffmpeg.c: In function ‘send_frame_to_filters’:
src/fftools/ffmpeg.c:2257: warning: ‘ret’ may be used uninitialized in this function
LD      ffmpeg_g
gcc: fftools/ffmpeg_filter.o: No such file or directory
gcc: fftools/ffmpeg_hw.o: No such file or directory
make: *** [ffmpeg_g] Error 1

Following each failure up with a second

$ make -j4 ffmpeg_g

completes the build successfully.

Apparently, some dependency is incorrect. I'm too stupid to debug it on my own though.

NOTE: The exact above will currently (git master as of 84e7aff60816d8f491194a9d96d906c39ad052db) fail for another reason, which needs to be fixed first (I will not open a ticket, but I have directly sent a patch to ffmpeg-devel ​http://ffmpeg.org/pipermail/ffmpeg-devel/2019-February/240065.html, ​https://patchwork.ffmpeg.org/patch/12052/): In ./configure, both "mlp_encoder_select" and "truehd_encoder_select" require the additional dependency "audio_frame_queue". With vanilla (i.e. unfixed) ffmpeg, you can probably omit the "truehd" from configure above, and achieve the same result which is the point of this ticket.

Note: This happens both on x86 (32 bit) with old gcc, make and glibc, as well as on modern x86_64 (F29 with all latest updates).

Other variations of the ./configure line also produce the same result, but this is the one I have right now. Also "-j3", "-j4", and "-j8" often result in the same (possibly with failures at different files / points in build).

[Carl Eugen Hoyos]

Is make clean necessary to reproduce?

[Carl Eugen Hoyos]

Not a duplicate of #6939, not a regression afaict but I am not convinced that there is an issue.

If there is a bug, it is reproducible with ./configure --disable-everything --disable-network --disable-vdpau --disable-asm && make -j8 clean ffmpeg_g

[mkver]

Looking at the Makefile, I can't say I'm surprised by what you experience. There is just nothing that synchronizes that the clean target is executed before rebuilding everything, so executing clean may remove files just created.

I actually never had the expectation that adding another target to clean should work at all; I always use "make clean && make <other targets>". So I don't really consider this a bug.

[Elon Musk]

bug by design