Opened 2 years ago

Last modified 13 months ago

#6638 new defect

aresample does not work for unknown channel layouts

Reported by: CoRoNe Owned by:
Priority: normal Component: avfilter
Version: git-master Keywords: aresample
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

FFmpeg used:

ffmpeg version N-86763-g341f012-Reino Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.1.0 (GCC)
  configuration: --arch=x86 --target-os=mingw32 --cross-prefix=/cygdrive/m/ffmpeg-windows-build-helpers-master/native_build/windows/ffmpeg_local_builds/sandbox/cross_compilers/mingw-w64-i686/bin/i686-w64-mingw32- --pkg-config=pkg-config --pkg-config-flags=--static --extra-version=Reino --enable-gray --enable-version3 --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-w32threads --disable-schannel --enable-fontconfig --enable-gmp --enable-gnutls --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfdk-aac --enable-libflite --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libzimg --enable-libzvbi --enable-gpl --enable-avisynth --enable-frei0r --enable-filter=frei0r --enable-librubberband --enable-libvidstab --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-avresample --extra-cflags='-march=pentium3' --extra-cflags=-O2 --extra-cflags='-mfpmath=sse' --extra-cflags=-msse --enable-static --disable-shared --prefix=/cygdrive/m/ffmpeg-windows-build-helpers-master/native_build/windows/ffmpeg_local_builds/sandbox/cross_compilers/mingw-w64-i686/i686-w64-mingw32
  libavutil      55. 67.100 / 55. 67.100
  libavcodec     57.100.104 / 57.100.104
  libavformat    57. 75.100 / 57. 75.100
  libavdevice    57.  7.100 / 57.  7.100
  libavfilter     6. 95.100 /  6. 95.100
  libavresample   3.  6.  0 /  3.  6.  0
  libswscale      4.  7.101 /  4.  7.101
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100

The last couple of days I've been experimenting with the loudnorm-filter, but only to find out that something fishy is going on with the aresample-filter:

ffmpeg -v 40 -i input.wav -af "loudnorm,aresample=44100" -f null NUL
[graph_0_in_0_0 @ 04bc07a0] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:0x3
[Parsed_loudnorm_0 @ 04b29540] auto-inserting filter 'auto_resampler_0' between the filter 'graph_0_in_0_0' and the filter 'Parsed_loudnorm_0'
[Parsed_aresample_1 @ 04bc0360] Cannot select channel layout for the link between filters Parsed_aresample_1 and format_out_0_0.
Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #0:0

You'd explicitly have to use "aformat=channel_layouts=stereo" or "-ac 2", or it will fail.

ffmpeg -v 40 -i input.wav -af "loudnorm,aresample=44100,aformat=channel_layouts=stereo" -f null NUL
[graph_0_in_0_0 @ 04bc04a0] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:0x3
[Parsed_loudnorm_0 @ 04b29560] auto-inserting filter 'auto_resampler_0' between the filter 'graph_0_in_0_0' and the filter 'Parsed_loudnorm_0'
[auto_resampler_0 @ 04bd2400] ch:2 chl:stereo fmt:s16 r:44100Hz -> ch:2 chl:stereo fmt:dbl r:192000Hz
[Parsed_aresample_1 @ 04b754c0] ch:2 chl:stereo fmt:dbl r:192000Hz -> ch:2 chl:stereo fmt:s16 r:44100Hz
Output #0, null, to 'NUL':
    Stream #0:0: Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s

For the video I'm trying to encode I use the adelay-filter next. Strange thing is that somehow in this case "aformat=channel_layouts=stereo" isn't needed:

ffmpeg -v 40 -i input.wav -af "loudnorm,aresample=44100,adelay=1000|1000" -f null NUL
[graph_0_in_0_0 @ 04bb0b40] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:0x3
[Parsed_loudnorm_0 @ 04b29520] auto-inserting filter 'auto_resampler_0' between the filter 'graph_0_in_0_0' and the filter 'Parsed_loudnorm_0'
[format_out_0_0 @ 04bd22c0] auto-inserting filter 'auto_resampler_1' between the filter 'Parsed_adelay_2' and the filter 'format_out_0_0'
[auto_resampler_0 @ 04ba0a00] ch:2 chl:stereo fmt:s16 r:44100Hz -> ch:2 chl:stereo fmt:dbl r:192000Hz
[Parsed_aresample_1 @ 04bc0320] ch:2 chl:stereo fmt:dbl r:192000Hz -> ch:2 chl:stereo fmt:dblp r:44100Hz
[auto_resampler_1 @ 04d39ee0] ch:2 chl:stereo fmt:dblp r:44100Hz -> ch:2 chl:stereo fmt:s16 r:44100Hz
Output #0, null, to 'NUL':
    Stream #0:0: Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s

But it gets weirder. The following encoders (that I've tested) seem to accept the adelay-filter's output:

ffmpeg -v 40 -i input.wav -af "loudnorm,aresample=44100,adelay=1000|1000" -c:a {libfdk_aac,libmp3lame,libtwolame,ac3,flac,alac,tta} -f null NUL
[graph_0_in_0_0 @ 04bc0220] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:0x3
[graph_0_in_0_0 @ 04bc0520] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:0x3
[Parsed_loudnorm_0 @ 04baf760] auto-inserting filter 'auto_resampler_0' between the filter 'graph_0_in_0_0' and the filter 'Parsed_loudnorm_0'
[format_out_0_0 @ 04ba0ba0] auto-inserting filter 'auto_resampler_1' between the filter 'Parsed_adelay_2' and the filter 'format_out_0_0'
[auto_resampler_0 @ 04b91760] ch:2 chl:stereo fmt:s16 r:44100Hz -> ch:2 chl:stereo fmt:dbl r:192000Hz
[Parsed_aresample_1 @ 04b29580] ch:2 chl:stereo fmt:dbl r:192000Hz -> ch:2 chl:stereo fmt:dblp r:44100Hz
[auto_resampler_1 @ 04d39e40] ch:2 chl:stereo fmt:dblp r:44100Hz -> ch:2 chl:stereo fmt:s16 r:44100Hz
Output #0, null, to 'NUL':
    Stream #0:0: Audio: aac (libfdk_aac), 44100 Hz, stereo, s16, delay 2048, 128 kb/s
    Stream #0:0: Audio: mp3 (libmp3lame), 44100 Hz, stereo, fltp, delay 1105
    Stream #0:0: Audio: mp2 (libtwolame), 44100 Hz, stereo, fltp, delay 481, 384 kb/s
    Stream #0:0: Audio: ac3, 44100 Hz, stereo, fltp, delay 256, 192 kb/s
    Stream #0:0: Audio: flac, 44100 Hz, stereo, s32 (24 bit), 128 kb/s
    Stream #0:0: Audio: alac, 44100 Hz, stereo, s32p (24 bit), 128 kb/s
    Stream #0:0: Audio: tta, 44100 Hz, stereo, s32 (24 bit), 128 kb/s

But the following encoders (that I've tested) don't:

ffmpeg -v 40 -i input.wav -af "loudnorm,aresample=44100,adelay=1000|1000" -c:a {libopus,libvorbis,aac,wmav2,wavpack} -f null NUL
[graph_0_in_0_0 @ 04bc0520] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:0x3
[Parsed_loudnorm_0 @ 04b29580] auto-inserting filter 'auto_resampler_0' between the filter 'graph_0_in_0_0' and the filter 'Parsed_loudnorm_0'
[format_out_0_0 @ 04ba0ba0] auto-inserting filter 'auto_resampler_1' between the filter 'Parsed_adelay_2' and the filter 'format_out_0_0'
[Parsed_aresample_1 @ 04cdb420] Cannot select channel layout for the link between filters Parsed_aresample_1 and Parsed_adelay_2.
Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #0:0

They again need "aformat=channel_layouts=stereo" or "-ac 2".

ffmpeg -v 40 -i input.wav -af "loudnorm,aresample=44100,aformat=channel_layouts=stereo,adelay=1000|1000" -c:a {libopus,libvorbis,aac,wmav2,wavpack} -f null NUL
[graph_0_in_0_0 @ 04baf7e0] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:0x3
[Parsed_loudnorm_0 @ 04b29620] auto-inserting filter 'auto_resampler_0' between the filter 'graph_0_in_0_0' and the filter 'Parsed_loudnorm_0'
[format_out_0_0 @ 04b91f40] auto-inserting filter 'auto_resampler_1' between the filter 'Parsed_adelay_3' and the filter 'format_out_0_0'
[auto_resampler_0 @ 04b83820] ch:2 chl:stereo fmt:s16 r:44100Hz -> ch:2 chl:stereo fmt:dbl r:192000Hz
[Parsed_aresample_1 @ 04b906a0] ch:2 chl:stereo fmt:dbl r:192000Hz -> ch:2 chl:stereo fmt:dblp r:44100Hz
[auto_resampler_1 @ 04bc0040] ch:2 chl:stereo fmt:dblp r:44100Hz -> ch:2 chl:stereo fmt:flt r:48000Hz
Output #0, null, to 'NUL':
    Stream #0:0: Audio: opus (libopus), 48000 Hz, stereo, flt, delay 312, 96 kb/s
    Stream #0:0: Audio: vorbis (libvorbis), 44100 Hz, stereo, fltp
    Stream #0:0: Audio: aac (LC), 44100 Hz, stereo, fltp, delay 1024, 128 kb/s
    Stream #0:0: Audio: wmav2, 44100 Hz, stereo, fltp, delay 2048, 128 kb/s
    Stream #0:0: Audio: wavpack, 44100 Hz, stereo, fltp, 128 kb/s

So it seems (if my judgement is correct) that not only aresample, but also adelay doesn't work for unknown channel layouts.

I guess this issue is quite similar to https://trac.ffmpeg.org/ticket/5218.

Change History (5)

comment:1 Changed 20 months ago by richardpl

adelay works with unknown channel layouts, you will see that if you run it solely for filtering audio.

comment:2 Changed 19 months ago by CoRoNe

  • Version changed from unspecified to git-master

Today I've had another look at this. (FFmpeg used: ffmpeg-N-90853-g3914a76)

ffmpeg -v 40 -f lavfi -i sine -af "loudnorm" -f null NUL
[...]
[graph_0_in_0_0 @ 05b35e00] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:0x4
[Parsed_loudnorm_0 @ 05b35880] auto-inserting filter 'auto_resampler_0' between the filter 'graph_0_in_0_0' and the filter 'Parsed_loudnorm_0'
[format_out_0_0 @ 05b36dc0] auto-inserting filter 'auto_resampler_1' between the filter 'Parsed_loudnorm_0' and the filter 'format_out_0_0'
[auto_resampler_0 @ 05b37ac0] ch:1 chl:mono fmt:s16 r:44100Hz -> ch:1 chl:mono fmt:dbl r:192000Hz
[auto_resampler_1 @ 05b37600] ch:1 chl:mono fmt:dbl r:192000Hz -> ch:1 chl:mono fmt:s16 r:192000Hz

To mimic the same output (sample-format and sample-rate wise):

ffmpeg -v 40 -f lavfi -i sine -af "aformat=dbl:192000" -f null NUL
[...]
[graph_0_in_0_0 @ 05b36040] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:0x4
[Parsed_aformat_0 @ 05b358c0] auto-inserting filter 'auto_resampler_0' between the filter 'graph_0_in_0_0' and the filter 'Parsed_aformat_0'
[format_out_0_0 @ 05b36dc0] auto-inserting filter 'auto_resampler_1' between the filter 'Parsed_aformat_0' and the filter 'format_out_0_0'
[auto_resampler_0 @ 05b37900] ch:1 chl:mono fmt:s16 r:44100Hz -> ch:1 chl:mono fmt:dbl r:192000Hz
[auto_resampler_1 @ 05b37440] ch:1 chl:mono fmt:dbl r:192000Hz -> ch:1 chl:mono fmt:s16 r:192000Hz

Adding aresample to the filter chain shows the same error as with the loudnorm filter:

ffmpeg -v 40 -f lavfi -i sine -af "aformat=dbl:192000,aresample=44100" -f null NUL
[...]
[graph_0_in_0_0 @ 05b4ab40] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:0x4
[Parsed_aformat_0 @ 05b358c0] auto-inserting filter 'auto_resampler_0' between the filter 'graph_0_in_0_0' and the filter 'Parsed_aformat_0'
[Parsed_aresample_1 @ 05b360c0] Cannot select channel layout for the link between filters Parsed_aresample_1 and format_out_0_0.
Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #0:0

This can be fixed with ocl=mono:

ffmpeg -v 40 -f lavfi -i sine -af "aformat=dbl:192000,aresample=44100:ocl=mono" -f null NUL
[...]
[graph_0_in_0_0 @ 05b4ac40] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:0x4
[Parsed_aformat_0 @ 05b358c0] auto-inserting filter 'auto_resampler_0' between the filter 'graph_0_in_0_0' and the filter 'Parsed_aformat_0'
[auto_resampler_0 @ 05b4c980] ch:1 chl:mono fmt:s16 r:44100Hz -> ch:1 chl:mono fmt:dbl r:192000Hz
[Parsed_aresample_1 @ 05b360c0] ch:1 chl:mono fmt:dbl r:192000Hz -> ch:1 chl:mono fmt:s16 r:44100Hz

...but the strange thing is that it appears the channel layout only gets lost with this particular format (double, 192000Hz), because ocl=mono isn't needed when I only change the sample-format or the sample-rate separately:

ffmpeg -v 40 -f lavfi -i sine -af "aformat=sample_rates=192000,aresample=44100" -f null NUL
[...]
[graph_0_in_0_0 @ 05b4aa00] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:0x4
[Parsed_aformat_0 @ 05b358c0] auto-inserting filter 'auto_resampler_0' between the filter 'graph_0_in_0_0' and the filter 'Parsed_aformat_0'
[auto_resampler_0 @ 05b4c800] ch:1 chl:mono fmt:s16 r:44100Hz -> ch:1 chl:mono fmt:s16 r:192000Hz
[Parsed_aresample_1 @ 05b35e80] ch:1 chl:mono fmt:s16 r:192000Hz -> ch:1 chl:mono fmt:s16 r:44100Hz
ffmpeg -v 40 -f lavfi -i sine -af "aformat=sample_fmts=dbl,aresample=44100" -f null NUL
[...]
[graph_0_in_0_0 @ 05b4a9c0] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:0x4
[Parsed_aformat_0 @ 05b358c0] auto-inserting filter 'auto_resampler_0' between the filter 'graph_0_in_0_0' and the filter 'Parsed_aformat_0'
[auto_resampler_0 @ 05b4c780] ch:1 chl:mono fmt:s16 r:44100Hz -> ch:1 chl:mono fmt:dbl r:44100Hz
[Parsed_aresample_1 @ 05b35e80] ch:1 chl:mono fmt:dbl r:44100Hz -> ch:1 chl:mono fmt:s16 r:44100Hz
Last edited 19 months ago by CoRoNe (previous) (diff)

comment:3 Changed 18 months ago by CoRoNe

Can anyone reproduce and have an explanation?

comment:4 Changed 13 months ago by zaterio

Similar behavior in last commit (Nov 2018):

[root@localhost 2]# /usr/local/ffmpeg-master-nvenc-fdk-aac/bin/ffmpeg -buildconf
ffmpeg version N-92394-g75625c5 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-16)
  configuration: --prefix=/usr/local/ffmpeg-master-nvenc-fdk-aac --pkg-config-flags=--static --extra-cflags=-I/usr/src/ffmpeg_build/include --extra-cflags=-I/usr/local/cuda-9.2/include/ --extra-ldflags=-L/usr/local/cuda-9.2/lib64/ --extra-ldflags=-L/usr/src/ffmpeg_build/lib --extra-libs=-lpthread --extra-libs=-lm --enable-gpl --enable-libx264 --enable-nonfree --enable-cuda --enable-cuvid --enable-nvenc --enable-libnpp --enable-libfdk_aac
  libavutil      56. 23.101 / 56. 23.101
  libavcodec     58. 39.100 / 58. 39.100
  libavformat    58. 22.100 / 58. 22.100
  libavdevice    58.  6.100 / 58.  6.100
  libavfilter     7. 43.100 /  7. 43.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100

  configuration:
    --prefix=/usr/local/ffmpeg-master-nvenc-fdk-aac
    --pkg-config-flags=--static
    --extra-cflags=-I/usr/src/ffmpeg_build/include
    --extra-cflags=-I/usr/local/cuda-9.2/include/
    --extra-ldflags=-L/usr/local/cuda-9.2/lib64/
    --extra-ldflags=-L/usr/src/ffmpeg_build/lib
    --extra-libs=-lpthread
    --extra-libs=-lm
    --enable-gpl
    --enable-libx264
    --enable-nonfree
    --enable-cuda
    --enable-cuvid
    --enable-nvenc
    --enable-libnpp
    --enable-libfdk_aac

# MPEGTS test

/usr/local/ffmpeg-master-nvenc-fdk-aac/bin/ffmpeg -loglevel debug \
-hwaccel cuvid -c:v h264_cuvid \
-y -i /home/trz/content/profile-abr_progresive_hp_r30_hls/avengers60fps.mp4  \
-filter_complex "[v:0]split=2[s0][s1];[0:a]asplit=2[ao1][ao2]" \
-map [s0] -map [ao1] -c:v h264_nvenc -preset hp -b:v 3000k -rc cbr -c:a libfdk_aac -b:a 96k -ar 48.0k -ac 2 \
-f mpegts 3000.ts \
-map [s1] -map [ao2] -c:v h264_nvenc -preset hp -b:v 1000k -rc cbr -c:a libfdk_aac -b:a 96k -ar 48.0k -ac 2 \
-f mpegts 1000.ts 2> out1.log
[root@localhost 2]# mediainfo 1000.ts 
General
ID                                       : 1 (0x1)
Complete name                            : 1000.ts
Format                                   : MPEG-TS
File size                                : 104 MiB
Duration                                 : 11 min 20 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 1 277 kb/s
[root@localhost 2]# mediainfo 3000.ts 
General
ID                                       : 1 (0x1)
Complete name                            : 3000.ts
Format                                   : MPEG-TS
File size                                : 271 MiB
Duration                                 : 11 min 20 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 3 337 kb/s

# HLS test

/usr/local/ffmpeg-master-nvenc-fdk-aac/bin/ffmpeg -loglevel debug \
-hwaccel cuvid -c:v h264_cuvid \
-y -i /home/trz/content/profile-abr_progresive_hp_r30_hls/avengers60fps.mp4  \
-filter_complex "[v:0]split=2[s0][s1];[0:a]asplit=2[ao1][ao2]" \
-map [s0] -map [ao1] -c:v h264_nvenc -preset hp -b:v 3000k -rc cbr -c:a libfdk_aac -b:a 96k -ar 48.0k -ac 2 \
-map [s1] -map [ao2] -c:v h264_nvenc -preset hp -b:v 1000k -rc cbr -c:a libfdk_aac -b:a 96k -ar 48.0k -ac 2 \
-f hls \
-var_stream_map "v:0,a:0 v:1,a:1" -hls_list_size 0 -hls_time 9 \
-master_pl_name master.m3u8 vs%v/manifest.m3u8 2> out2.log
[root@localhost 2]# cat master.m3u8 
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=1205600,RESOLUTION=1920x1080,CODECS="avc1.4d402a,mp4a.40.2"
vs0/manifest.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=1205600,RESOLUTION=1920x1080,CODECS="avc1.4d402a,mp4a.40.2"
vs1/manifest.m3u8
[root@localhost 2]# mediainfo vs0/manifest15.ts 
General
ID                                       : 1 (0x1)
Complete name                            : vs0/manifest15.ts
CompleteName_Last                        : vs0/manifest75.ts
Format                                   : MPEG-TS
File size                                : 79.2 MiB
Duration                                 : 9 min 3 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 1 222 kb/s
[root@localhost 2]# mediainfo vs1/manifest15.ts 
General
ID                                       : 1 (0x1)
Complete name                            : vs1/manifest15.ts
CompleteName_Last                        : vs1/manifest75.ts
Format                                   : MPEG-TS
File size                                : 79.2 MiB
Duration                                 : 9 min 3 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 1 222 kb/s
Version 0, edited 13 months ago by zaterio (next)

comment:5 Changed 13 months ago by cehoyos

  • Keywords unknown channel layouts removed
Note: See TracTickets for help on using tickets.