ffmpeg not remapping channels for libopus automatically

[agressiv]

[libopus @ 00000000026ad020] Invalid channel layout 5.1(side) for specified mapping family -1.

Summary of the bug:
Encode a 5.1 audio file with 5.1(side) channel map
How to reproduce:

% ffmpeg -i movie-with-51side.mkv -c:a libopus -b:a 256k output.mkv
ffmpeg version N-81025-g25ca74d
built on July 15 2016

ffmpeg version N-80101-gd970f7b automatically remapped these to the rear channels since libopus doesn't seem to allow side channels. I now have to explicitly add:

-af "channelmap=channel_layout=5.1"

Not sure if this is expected behavior or not. Without this channelmap, it aborts. Command-line opusenc automatically remaps as well.

[Hendrik]

Probably a regression since 37941878f193a2316c514bd5ba55bfe9d2dfdfcf. Removing the supported channel_layouts from the codec definition was really not a good idea for on obscure feature.

[James]

Definitely a regression, and indeed started by 37941878f193a2316c514bd5ba55bfe9d2dfdfcf.

New defaults shouldn't make a simple "ffmpeg -i INPUT -c:a libopus OUTPUT" start failing.

[Michael Niedermayer]

Mailed author of 37941878f193a2316c514bd5ba55bfe9d2dfdfcf

[Michael Graczyk]

When I wrote this patch, I did not realize that channel_layouts was used to automatically remix the channel layout from 5.1(side) to 5.1. I did not intend to change default behavior with that patch.

However, my understanding was that the error here is potentially desirable. The ogg audio stream in output.mkv has a different channel layout than the audio in the input file.

I see that in most cases remapping 5.1(side) to 5.1 is the right thing to do. Is there any way to get this automatic remapping only when the mapping_family parameter is not provided?

Also, what do you mean by "Command-line opusenc automatically remaps as well"? How do you process mkv files with opusenc? Are you sure the process used to extract audio from the mkv file didn't do the remapping?

[agressiv]

I will pipe it from eac3to. opusenc can take piped input.

eac3to.exe source.mkv stdout.wav | opusenc.exe --bitrate 256k --ignorelength - out.opus

WARNING: WAV file uses side surround instead of rear for 5.1;
remapping side speakers to rear in encoding.
Encoding using libopus 1.1 (audio)

[Michael Graczyk]

agressiv, thanks for that. I see that opusenc has a few special cases for moving rear to side and side to rear. I would have liked to preserve the behavior in this case as it was before 37941878f by default. I am hopeful we can restore this behavior without removing the mapping_family argument.

All,
Would it make sense to add these special cases explicitly to libopusenc.c, and log a warning as is done by opusenc?

[Hendrik]

I would prefer if opusenc had the channel_layouts setting again so that ffmpeg.c and other user apps know which layouts opus supports and can decide based on that which layout to use and potentially remix to, instead of hoping opusenc supports that internally.

channel_layouts is not only meant to limit the input to an encoder, its also information for the calling code so it can pick the best format on its own and not error out later.

[Michael Graczyk]

heleppkes,

libopusenc can accept many channel layouts that are not possible to express using the channel_layouts bitfield system. There are channel layouts which are supported by libopus which are not expressible by the channel_layouts member. I removed channel_layouts from the codec definition to prevent ffmpeg.c from halting with an error when one of these unexpressible layouts was passed in.

Is there a way to specify at the command line "no channel layout", or "unknown layout"? With such a flag, I could add channel_layouts back to libopusenc and also add code to handle the "no layout" case. I believe last time I asked about this, it was problematic because the channel count is represented by the popcount of the layout bitfield, and the "no channel layout" layout could have any channel count.

[Peter White]

Reenable channel_layouts

[Peter White]

I just added a quick patch to get channel_layouts back. I just got bitten by this as well.

@mgraczyk I don't exactly understand what you are talking about when saying libopusenc can accept many inexpressible layouts. But is it more likely to encounter those than the most prominent ones, like 5.1(side)? I think for the time being, adding channel_layouts back in is the better compromise.

Anyway, the attached patch is there for anyone wanting to workaround this issue. To be honest, though, I did not check any implications, since I wouldn't know how, I am not a developer. But adding this one line back in seems easy enough. Correct me if I am wrong.

[Peter White]

A more general workaround, without patching is to use the aformat filter, like i.e.:
ffmpeg -i surround_input -af aformat=channel_layouts="7.1|5.1|stereo" output.opus
This way one does not need to manually set an appropriate channel map with differing input layouts and channel numbers. Put this in a script and select the relevant layouts. The above example is just a quick one.

[Federico]

The issue is still present in the latest nightly.
The only thing that seems to have been fixed is that all workarounds no longer work.

It's been almost TWO YEARS. This is a serious limitation!

[Brasrok]

Just wanted to push this issue. As dosse91 wrote ealier, none of the workarounds work anymore and this is a serious limitation, guys.

[mzso]

So, I guess this was decided to remain broken by design? Then why is it still open?

[Tom B]

This is still an issue in 4.1.3. Can libobus be used for 5.1 audio? I've been using libvorbis for now but would be preferable to opus.

[Soichiro]

Still present in 4.2.1. Could we at least get a response on this? Much thanks to the work by Peter to provide a patch and a workaround, but I would much like to see the patch merged so the workaround isn't needed.

[Christoph]

Still present (after 4 years) in version 4.3.1.

[everlanes]

I also ran into this problem and the suggested workaround is not working for me.

I want to convert MKV files with multiple audio channels to opus audio keeping the layout of every single channel. When using the -af channelmap=channel_layout="5.1" option I get an error message for all stereo channels:

[Parsed_channelmap_0 @ 0x55a7dde59300] input channel #2 not available from input layout 'stereo'
[Parsed_channelmap_0 @ 0x55a7dde59300] input channel #3 not available from input layout 'stereo'
[Parsed_channelmap_0 @ 0x55a7dde59300] input channel #4 not available from input layout 'stereo'
[Parsed_channelmap_0 @ 0x55a7dde59300] input channel #5 not available from input layout 'stereo'
[Parsed_channelmap_0 @ 0x55a7dde59300] Failed to configure input pad on Parsed_channelmap_0
Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #0:5

Furthermore I expect all 7.1 channels to be downmixed to 5.1 as well with these settings. I don't want that either.

When removing the filter I get the original error:

[libopus @ 0x563a3830d9c0] Invalid channel layout 5.1(side) for specified mapping family -1.
Error initializing output stream 0:2 -- Error while opening encoder for output stream #0:2 - maybe incorrect parameters such as bit_rate, rate, width or height

This seems to be a trivial problem as all it takes would be changing metadata 5.1(side) to (5.1). No downmixing, no recompressing of data, nothing. But yet, this bug is unresolved since five years now!

Is there a solution (or another workaround ...) for this situation?
Maybe a 3rd party tool or separate ffmpeg run just to change metadata of all 5.1(side) channels to 5.1?

[everlanes]

So I hacked together a quite complicated script to circumvent this issue (hopefully), but I am not very proud of it and I think it's strange I have to do it:

FILE="NameOfFileToConvert.mkv"

read -r -d "" PYTHONCODE << EOD
import sys, json 

data = json.load(sys.stdin)

replace = {
        "5.0(side)": "5.0",
        "5.1(side)": "5.1",
        # you can add more mappings here
    }

filter_string = "[:%d]channelmap=channel_layout='%s'" 
copy_string = "[:%d]anull"

maps = [filter_string % (stream["index"], replace[stream["channel_layout"]])
        if stream.get("channel_layout") in replace.keys() else
        copy_string % (stream["index"])
        for stream in data["streams"]
        if stream["codec_type"] == "audio"]

print(";".join(maps))
EOD

ffmpeg -i "$FILE" -map 0 \
    -c:v copy \
    -c:a libopus -filter_complex \
        `ffprobe -hide_banner -show_streams -print_format json "$FILE" | \
        python3 -c "$PYTHONCODE"` \
    -c:s copy \
    "$OUT_FILE"

So basically I get all streams of the original file and pipe it into a python script that creates the filter string that remaps all problematic streams. This string is then printed and used as an argument for the ffmpeg call...

Does it work? Hopefully. Is it nice? Definitely not! There should be some kind of an (optionally) fallback mechanism in ffmpeg and/or libopus for unsupported channel layouts.

[DesertCookie]

I am surprised this is still an issue with FFmpeg version 5 and 6.

[RüGi]

Just installed 'ffmpeg version N-111259-g4aa1a42a91-20230623' and this error still occurs.

There may be good reasons why it is not yet fixed, but then it is very strange for me that there is no comment or explanation, why it is not fixed.

I wonder why libopus does not know how to deal with channel_layout 5.1(side), since it is quite common in German DVB-T2 (digital terrestrial TV).

A comment of some 'wise' (wo)men would be appreciated!

P.S. For now, I can live with the solution form 'comment:11 by Peter White' (using

af aformat=channel_layouts="7.1|5.1|stereo

, but it is still an open question to me.
Thanks

[GoingOffRoading]

Still an issue in October 2023.

Confirmed that the workaround outline in the beginning of this thread no longer work

[Elliot Lee]

Running into this again - December 2023. comment:11 has the best workaround I've found.

[Elliot Lee]

So in order to check which channel layouts are allowed, libavcodec/libopusenc.c uses libavcodec/vorbis_data.c's array of ff_vorbis_ch_layouts. I think the root of the problem is that ff_vorbis_ch_layouts does not list AV_CHANNEL_LAYOUT_5POINT1 as a possible channel layout. Is there any reason why it's not there?

[Elliot Lee]

Patch libopus encoder to handle 5.1(side) channel layout.

[Elliot Lee]

The patch works for me. I encoded ​https://www.thelees.me/download/nasa.mp4 into ​https://www.thelees.me/download/nasa.webm without problem. The patch also seems to change ffprobe to recognize opus streams that are encoded with 5.1(side)...

[Elliot Lee]

Please don't use my previous patch. It's ill-advised and silently breaks stuff. For the short-term, the command-line workaround with aformat= is the best to use.

The root of the problem is that the Opus format (​https://www.rfc-editor.org/rfc/rfc7845.html#section-5.1.1 ) uses the Vorbis mapping family (​https://www.xiph.org/vorbis/doc/Vorbis_I_spec.html#x1-810004.3.9 ) for doing more complex channel layouts. Unfortunately, the existing scheme only represents a few common channel layouts, and does not allow more than one channel layout for any given channel count.

I think for the long term, what needs to be done is:

1. Define an Opus channel mapping family (​https://www.iana.org/assignments/opus-channel-mapping-families/opus-channel-mapping-families.xhtml) that can represent the rest of the AV_CHANNEL_LAYOUT_* channel layouts in ffmpeg/libavutil/channel_layout.h . ffmpeg is a good reference for this because it supports so many obscure channel layouts.
2. Get libopus to know how to handle this new channel mapping family.
3. Get ffmpeg using the new libopus to turn its internal channel layouts into the Opus channel layouts.

So really, at this point more of this is a libopus task than an ffmpeg task. Anyone have contacts with the opus-codec people that can incorporate this idea on their roadmap?

[Elliot Lee]

Just opened ​https://gitlab.xiph.org/xiph/opus/-/issues/2374

[MasterQuestionable]

͏    For [ ​https://gitlab.xiph.org/xiph/opus/-/issues/2374 ], but I don't have a GitLab account.

͏    ​https://trac.ffmpeg.org/ticket/11002#issuecomment-2053773784