x264 init race/corruption

[lukealonso]

Summary of the bug:

I have a program that creates many concurrent instances of x264 via AVCodecContext. Occasionally, I'll get the following output:

x264 [error]: invalid tune '[RANDOM JUNK]'

and avcodec_open2 will fail.

I've attached a stripped down program that reproduces the issue, at least on Ubuntu 16.04.

This appears to happen with all x264 versions, I tried the latest stable commit, and one from several years ago.

This does *NOT* happen with older FFmpeg versions. I git bisected and found that the problem started happening on commit:

[abaa12263e08ae7b2099f00fa4597ae6a86d7a9c] avcodec: Don't lock during open if the codec has threadsafe init]

.... which is not surprising, but not terribly helpful in finding what exactly is happening in X264_init that causes the issue.

Hopefully this is just me being dumb, any pointers would be appreciated.

How to reproduce:

Build enc_test.c:

gcc -o enc_test enc_test.c -lavcodec -lavformat -lavutil -lpthread

Run 1-10 times, with num_threads between 4-32, whatever is appropriate for your machine. Happens more often with higher thread counts. It should happen early on, so ctrl-c and try again if it hasn't happened with in a few seconds.

./enc_test [num_threads]

[lukealonso]

repro

[Carl Eugen Hoyos]

Please confirm that the issue is reproducible with current FFmpeg git head and current libx264 git head.

[lukealonso]

Replying to cehoyos:
Yes, it occurs with ffmpeg git head and libx264 git head.

[Carl Eugen Hoyos]

Reproducible with both old and current x264.

[Marton Balint]

Fixed in b02490a497009064b7f192802aa246aa0b6a4dad.