Opened 9 months ago

Last modified 8 months ago

#9502 open defect

Intermittent error when opening audio device with avfoundation (mac)

Reported by: toots Owned by: Thilo Borgmann
Priority: normal Component: avdevice
Version: unspecified Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

We are having intermittent issues when quickly opening/closing avfoundation.

The error that shows up:
`
[avfoundation @ 0x7f964f032200] audio format is not supported
`

The attached file can be used to reproduce the error. Please note that the original icecast connection details have been edited out.

Full log:
`
./ffmpeg
Starting ffmpeg connection test..
Input #0, avfoundation, from 'default':

Duration: N/A, start: 324489.404308, bitrate: 2822 kb/s
Stream #0:0: Audio: pcm_f32le, 44100 Hz, stereo, flt, 2822 kb/s

Output #0, mp3, to 'icecast':

Metadata:

TSSE : Lavf58.76.100

Stream #0:0: Audio: mp3 (libmp3lame), 44100 Hz, stereo, s32p, 128 kb/s

Reached max time!
[Parsed_astats_1 @ 0x7f964ec41b00] Channel: 1
[Parsed_astats_1 @ 0x7f964ec41b00] RMS level dB: -64.770078
[Parsed_astats_1 @ 0x7f964ec41b00] Channel: 2
[Parsed_astats_1 @ 0x7f964ec41b00] RMS level dB: -64.770078
[Parsed_astats_1 @ 0x7f964ec41b00] Overall
[libmp3lame @ 0x7f964f875000] 4 frames left in the queue on closing
One iteration done..
Input #0, avfoundation, from 'default':

Duration: N/A, start: 324492.892676, bitrate: 2822 kb/s
Stream #0:0: Audio: pcm_f32le, 44100 Hz, stereo, flt, 2822 kb/s

Output #0, mp3, to 'icecast':

Metadata:

TSSE : Lavf58.76.100

Stream #0:0: Audio: mp3 (libmp3lame), 44100 Hz, stereo, s32p, 128 kb/s

Reached max time!
[Parsed_astats_1 @ 0x7f964ec5ee40] Channel: 1
[Parsed_astats_1 @ 0x7f964ec5ee40] RMS level dB: -64.933917
[Parsed_astats_1 @ 0x7f964ec5ee40] Channel: 2
[Parsed_astats_1 @ 0x7f964ec5ee40] RMS level dB: -64.933917
[Parsed_astats_1 @ 0x7f964ec5ee40] Overall
[libmp3lame @ 0x7f965100a800] 4 frames left in the queue on closing
One iteration done..
[avfoundation @ 0x7f964f032200] audio format is not supported
errror: Input/output error
One iteration done..
`

Version: 4.4.1

Attachments (3)

ffmpeg.c (11.8 KB ) - added by toots 9 months ago.
Reproduction code.
24unpacked.patch (809 bytes ) - added by Thilo Borgmann 9 months ago.
use-audio-converter.patch (12.2 KB ) - added by toots 9 months ago.
Use AudioConverter API

Download all attachments as: .zip

Change History (11)

by toots, 9 months ago

Attachment: ffmpeg.c added

Reproduction code.

comment:1 by Thilo Borgmann, 9 months ago

Owner: set to Thilo Borgmann
Status: newopen

Can you change your libavdevice/avfoundation.m:734 ff to print out the format you receive?

comment:2 by toots, 9 months ago

Here are the relevant variables:

basic_desc->mFormatI: 'lpcm',
basic_desc->mChannelsPerFrame: 2,
basic_desc->mBitsPerChannel: 24,
basic_desc->mFormatFlags: 0x4 (kAudioFormatFlagIsSignedInteger)

It looks like the issue here is that the kAudioFormatFlagIsPacked flag isn't set.

Let us know if you have any suggestion, happy to hack/test further!

by Thilo Borgmann, 9 months ago

Attachment: 24unpacked.patch added

comment:3 by Thilo Borgmann, 9 months ago

You might try applying the 24unpacked.patch. Since I cannot reproduce on my machine, it's a shot into the blue, though.

Check if audio_non_interleaved flag changed its value as well.

comment:4 by toots, 9 months ago

Unfortunately, that only resulted in corrupted audio.

I'm looking at an approach using CoreAudio's AudioConverter. That would also take care of non-interleaved audio and could potentially be extended to support any audio format returned by avfoundation.

by toots, 9 months ago

Attachment: use-audio-converter.patch added

Use AudioConverter API

comment:5 by toots, 9 months ago

use-audio-converter.patch fixes the issue and extends to virtually any encoding supported/returned by the OS. Will be testing it a bit more but it looks like it's doing the job.

comment:6 by toots, 9 months ago

Okay, turned out support for non-interleaved to interleaved conversion was more tricky that I thought. I've updated the changes and also fixed a concurrency issue in the thread model.

I've pushed all the changes here if you want to apply them: https://github.com/FFmpeg/FFmpeg/pull/374

comment:7 by toots, 9 months ago

Scratch that, sorry for the noise! I'm working on a better re-implementation using atomic queues. I think I'll go with the usual review process b/c it's gonna be a pretty big change.

comment:8 by toots, 8 months ago

Hi! I submitted my patch fixing this on the mailing list: http://ffmpeg.org/pipermail/ffmpeg-devel/2021-November/288608.html. It's the first of 3, the two other ones are unrelated but one fixes an issues with frames being dropped.

Are you able to review it there?

Note: See TracTickets for help on using tickets.