Intermittent error when opening audio device with avfoundation (mac)

[toots]

We are having intermittent issues when quickly opening/closing avfoundation.

The error that shows up:
`
[avfoundation @ 0x7f964f032200] audio format is not supported
`

The attached file can be used to reproduce the error. Please note that the original icecast connection details have been edited out.

Full log:
`
./ffmpeg
Starting ffmpeg connection test..
Input #0, avfoundation, from 'default':

Duration: N/A, start: 324489.404308, bitrate: 2822 kb/s
Stream #0:0: Audio: pcm_f32le, 44100 Hz, stereo, flt, 2822 kb/s

Output #0, mp3, to 'icecast':

Metadata:

TSSE : Lavf58.76.100

Stream #0:0: Audio: mp3 (libmp3lame), 44100 Hz, stereo, s32p, 128 kb/s

Reached max time!
[Parsed_astats_1 @ 0x7f964ec41b00] Channel: 1
[Parsed_astats_1 @ 0x7f964ec41b00] RMS level dB: -64.770078
[Parsed_astats_1 @ 0x7f964ec41b00] Channel: 2
[Parsed_astats_1 @ 0x7f964ec41b00] RMS level dB: -64.770078
[Parsed_astats_1 @ 0x7f964ec41b00] Overall
[libmp3lame @ 0x7f964f875000] 4 frames left in the queue on closing
One iteration done..
Input #0, avfoundation, from 'default':

Duration: N/A, start: 324492.892676, bitrate: 2822 kb/s
Stream #0:0: Audio: pcm_f32le, 44100 Hz, stereo, flt, 2822 kb/s

Output #0, mp3, to 'icecast':

Metadata:

TSSE : Lavf58.76.100

Stream #0:0: Audio: mp3 (libmp3lame), 44100 Hz, stereo, s32p, 128 kb/s

Reached max time!
[Parsed_astats_1 @ 0x7f964ec5ee40] Channel: 1
[Parsed_astats_1 @ 0x7f964ec5ee40] RMS level dB: -64.933917
[Parsed_astats_1 @ 0x7f964ec5ee40] Channel: 2
[Parsed_astats_1 @ 0x7f964ec5ee40] RMS level dB: -64.933917
[Parsed_astats_1 @ 0x7f964ec5ee40] Overall
[libmp3lame @ 0x7f965100a800] 4 frames left in the queue on closing
One iteration done..
[avfoundation @ 0x7f964f032200] audio format is not supported
errror: Input/output error
One iteration done..
`

Version: 4.4.1

[toots]

Reproduction code.

[Thilo Borgmann]

Can you change your libavdevice/avfoundation.m:734 ff to print out the format you receive?

[toots]

Here are the relevant variables:

basic_desc->mFormatI: 'lpcm',
basic_desc->mChannelsPerFrame: 2,
basic_desc->mBitsPerChannel: 24,
basic_desc->mFormatFlags: 0x4 (kAudioFormatFlagIsSignedInteger)

It looks like the issue here is that the kAudioFormatFlagIsPacked flag isn't set.

Let us know if you have any suggestion, happy to hack/test further!

[Thilo Borgmann]

You might try applying the 24unpacked.patch. Since I cannot reproduce on my machine, it's a shot into the blue, though.

Check if audio_non_interleaved flag changed its value as well.

[toots]

Unfortunately, that only resulted in corrupted audio.

I'm looking at an approach using CoreAudio's AudioConverter. That would also take care of non-interleaved audio and could potentially be extended to support any audio format returned by avfoundation.

[toots]

Use AudioConverter API

[toots]

use-audio-converter.patch fixes the issue and extends to virtually any encoding supported/returned by the OS. Will be testing it a bit more but it looks like it's doing the job.

[toots]

Okay, turned out support for non-interleaved to interleaved conversion was more tricky that I thought. I've updated the changes and also fixed a concurrency issue in the thread model.

I've pushed all the changes here if you want to apply them: ​https://github.com/FFmpeg/FFmpeg/pull/374

[toots]

Scratch that, sorry for the noise! I'm working on a better re-implementation using atomic queues. I think I'll go with the usual review process b/c it's gonna be a pretty big change.

[toots]

Hi! I submitted my patch fixing this on the mailing list: ​http://ffmpeg.org/pipermail/ffmpeg-devel/2021-November/288608.html. It's the first of 3, the two other ones are unrelated but one fixes an issues with frames being dropped.

Are you able to review it there?