FFmpeg -shortest not working when using -filter_complex with audio and video filters

[pigeonburger]

Pretty much the exact same issue was described ​here several years ago, but the 'solution' given, while it works, is more inconvenient and doesn't explain why the actual command in question does not work.

I'm using the following command to mix the audio of input.mp4 with song.m4a and ensure that the dimensions of my output file are even numbers:

ffmpeg -i input.mp4 -i song.m4a -filter_complex \
[0:a][1:a]amix[s0];[0:v]pad=height=ceil(ih/2)*2:width=ceil(iw/2)*2[s1] \
-map [s0] -map [s1] -shortest output.mp4

input.mp4 is only about 20 seconds long, but song.m4a is about 8 minutes long. I want to mix both the audio streams together, so that the first 20 seconds of song.m4a plays alongside the original audio of input.m4a. To ensure that the output file does not become 8 minutes long, I'm using the -shortest option, which should make FFmpeg stop encoding once it hits the shortest audio/video stream. However, in the command above, -shortest seems to be not working entirely.

When I separate the audio and video filters into 2 separate -filter_complexs, it works as intended (doing this was also the solution given in the issue a linked at the top):

ffmpeg -i input.mp4 -i song.m4a -filter_complex \
[0:a][1:a]amix[s0] -filter_complex [0:v]pad=height=ceil(ih/2)*2:width=ceil(iw/2)*2[s1] \
-map [s0] -map [s1] -shortest output.mp4

Same with if I remove the video filters entirely:

ffmpeg -i input.mp4 -i song.m4a -filter_complex [0:a][1:a]amix[s0] -map [s0] -map 0:v -shortest output.mp4

Using the above 2 commands results in a 20-second long video as intended.

I guess my question is why is -shortest made redundant when combining both audio and video filters in a single -filter_complex argument? Is this a bug, or the intended behaviour? How can I get it to work as intended with just one -filter_complex?

I'm using a self-compiled FFmpeg build on Linux (using the most recent Git version as of writing this) but I can confirm I can also recreate the issue with the official Linux AND Windows builds as well, even when using different audio and video files too.

---

The issue can be easily recreated by anyone using these commands (courtesy of Michael Koch):

// Create video file
ffmpeg -f lavfi -i testsrc2 -f lavfi -i sine=2000 -t 5 -y input.mp4

// Create song file
ffmpeg -f lavfi -i sine=500 -t 10 -y song.m4a

// Combine both files (-shortest should work here but it does not)
ffmpeg -i input.mp4 -i song.m4a -lavfi [0:a][1:a]amix[s0];[0:v]null[s1] -map [s0] -map [s1] -shortest -y output.mp4

[Michael Koch]

Might be related to ticket 8489, wrong audio length with amix filter.

[Michael Koch]

Does it work if you replace the *.mp4 input by *.mov? That's a workaround I found for ticket 8489.

[Michael Koch]

It seems the workaround with *.mov doesn't help in this case.

The issue can be reproduced with these simplified commands:

ffmpeg -f lavfi -i testsrc2 -f lavfi -i sine=2000 -t 5 -y input.mp4

ffmpeg -f lavfi -i sine=500 -t 10 -y song.m4a

ffmpeg -i input.mp4 -i song.m4a -lavfi [0:a][1:a]amix[s0];[0:v]null[s1] -map [s0] -map [s1] -shortest -y output.mp4

[Michael Koch]

The amix filter has a "duration" option. It works if you set this option to "shortest".
But this doesn't explain why the -shortest option doesn't work.

ffmpeg -i input.mp4 -i song.m4a -lavfi [0:a][1:a]amix=duration=shortest[s0];[0:v]null[s1] -map [s0] -map [s1] -y output.mp4

[pigeonburger]

Replying to Michael Koch:

The amix filter has a "duration" option. It works if you set this option to "shortest".
But this doesn't explain why the -shortest option doesn't work.

ffmpeg -i input.mp4 -i song.m4a -lavfi [0:a][1:a]amix=duration=shortest[s0];[0:v]null[s1] -map [s0] -map [s1] -y output.mp4

Even with that command -shortest is still not actually getting triggered when adding duration=shortest to amix, because all that's doing is shortening the m4a back to the original video's audio stream, which is likely the same length as the video itself, so it's still reverting to default behaviour. In that case it just appears to be working.

You can prove this by using an input video that has an audio stream shorter than the video stream and trying it with my original command. The m4a file will obey the duration=shortest and stop when the other audio stream finishes, but encoding will continue until the video stream, which in this case is the longest stream, stops. Of course, the expected behaviour here would be for FFmpeg to stop encoding once the shorter audio stream finishes, but that doesn't happen.

There's some sort of deep-seated long-standing bug here that's never been fixed and has been a known issue since at least 2014 (in the ticket I linked originally). I've tried having a look at the source code and debugging it myself but I'm not too experienced with C.