asf container - HE-AAC stream detected as "unknown: none"

[Djfe]

Summary of the bug:

ok, so I'm a student and I help at my university with video recordings.
They have a system in one of their buildings for their beamers to record all inputs and the microphones' audio.

The system outputs an asf container that contains 3 AVC streams and one HE-AAC audio stream.
FFmpeg isn't able to determine the codec type of the audio stream, yet.

It's possible to use FFmpeg to extract the audio with "-y -copy_unknown -f data" into an .m4a container
Once the stream is in an m4a container FFmpeg is able to determine the codec type.

I'll attach an affected file, so this can be improved in a future version of ffmpeg.

terminal output:

ffprobe '.\ASF Container - HE-AAC-Stream type unknown none.asf'
ffprobe version N-94030-gcaabe1b495 Copyright (c) 2007-2019 the FFmpeg developers
  built with gcc 8.3.1 (GCC) 20190414
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
  libavutil      56. 28.100 / 56. 28.100
  libavcodec     58. 53.100 / 58. 53.100
  libavformat    58. 27.103 / 58. 27.103
  libavdevice    58.  7.100 / 58.  7.100
  libavfilter     7. 55.100 /  7. 55.100
  libswscale      5.  4.101 /  5.  4.101
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
[asf @ 000002835cd2c200] Could not find codec parameters for stream 2 (Unknown: none): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, asf, from '.\ASF Container - HE-AAC-Stream type unknown none.asf':
  Metadata:
    WMFSDKNeeded    : 0.0.0.0000
    WMFSDKVersion   : 10.00.00.4066
    IsVBR           : 1
  Duration: 00:00:03.36, start: 0.016000, bitrate: 2238 kb/s
    Stream #0:0(chi): Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 1920x1080, 23 kb/s, 1k tbr, 1k tbn, 2k tbc
    Stream #0:1(chi): Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 1920x1080, 1743 kb/s, 1k tbr, 1k tbn, 2k tbc
    Stream #0:2(chi): Unknown: none
    Stream #0:3(chi): Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 1920x1080, 19 kb/s, 1k tbr, 1k tbn, 2k tbc
Unsupported codec with id 0 for input stream 2

ffmpeg -i '.\ASF Container - HE-AAC-Stream type unknown none.asf' -y -copy_unknown -map 0:2 -vn -c copy -f data audio.m4a
ffmpeg version N-94030-gcaabe1b495 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 8.3.1 (GCC) 20190414
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
  libavutil      56. 28.100 / 56. 28.100
  libavcodec     58. 53.100 / 58. 53.100
  libavformat    58. 27.103 / 58. 27.103
  libavdevice    58.  7.100 / 58.  7.100
  libavfilter     7. 55.100 /  7. 55.100
  libswscale      5.  4.101 /  5.  4.101
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
[asf @ 000002418e768f40] Could not find codec parameters for stream 2 (Unknown: none): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, asf, from '.\ASF Container - HE-AAC-Stream type unknown none.asf':
  Metadata:
    WMFSDKNeeded    : 0.0.0.0000
    WMFSDKVersion   : 10.00.00.4066
    IsVBR           : 1
  Duration: 00:00:03.36, start: 0.016000, bitrate: 2238 kb/s
    Stream #0:0(chi): Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 1920x1080, 23 kb/s, 1k tbr, 1k tbn, 2k tbc
    Stream #0:1(chi): Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 1920x1080, 1743 kb/s, 1k tbr, 1k tbn, 2k tbc
    Stream #0:2(chi): Unknown: none
    Stream #0:3(chi): Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 1920x1080, 19 kb/s, 1k tbr, 1k tbn, 2k tbc
Output #0, data, to 'audio.m4a':
  Metadata:
    WMFSDKNeeded    : 0.0.0.0000
    WMFSDKVersion   : 10.00.00.4066
    IsVBR           : 1
    encoder         : Lavf58.27.103
    Stream #0:0(chi): Unknown: none
Stream mapping:
  Stream #0:2 -> #0:0 (copy)
Press [q] to stop, [?] for help
[data @ 000002418f8693c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 234 >= 234
    Last message repeated 3 times
[data @ 000002418f8693c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 359 >= 359
[data @ 000002418f8693c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 484 >= 484
[data @ 000002418f8693c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 750 >= 750
[data @ 000002418f8693c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 875 >= 875
[data @ 000002418f8693c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1047 >= 1047
[data @ 000002418f8693c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1265 >= 1265
[data @ 000002418f8693c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1390 >= 1390
[data @ 000002418f8693c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1859 >= 1859
[data @ 000002418f8693c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1984 >= 1984
[data @ 000002418f8693c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2250 >= 2250
[data @ 000002418f8693c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2672 >= 2672
[data @ 000002418f8693c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2797 >= 2797
[data @ 000002418f8693c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2937 >= 2937
[data @ 000002418f8693c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 3093 >= 3093
size=      52kB time=00:00:03.29 bitrate= 129.2kbits/s speed= 207x
video:0kB audio:0kB subtitle:0kB other streams:52kB global headers:0kB muxing overhead: 0.000000%

ffprobe .\audio.m4a
ffprobe version N-94030-gcaabe1b495 Copyright (c) 2007-2019 the FFmpeg developers
  built with gcc 8.3.1 (GCC) 20190414
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
  libavutil      56. 28.100 / 56. 28.100
  libavcodec     58. 53.100 / 58. 53.100
  libavformat    58. 27.103 / 58. 27.103
  libavdevice    58.  7.100 / 58.  7.100
  libavfilter     7. 55.100 /  7. 55.100
  libswscale      5.  4.101 /  5.  4.101
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
[aac @ 0000021d8f1bc100] Estimating duration from bitrate, this may be inaccurate
Input #0, aac, from '.\audio.m4a':
  Duration: 00:00:03.33, bitrate: 128 kb/s
    Stream #0:0: Audio: aac (HE-AAC), 48000 Hz, stereo, fltp, 128 kb/s

My guess is that the demuxer isn't able to determine the codec type. Maybe aac in asf isn't even standardized. I actually don't know much about the asf container besides that it's from Microsoft.

The audio itself is mostly noise and some buzzing sound that is probably caused by the recording hardware.

If you need another larger sample file and/or someone talking on the mic, then just ask. I can provide those :)

[Carl Eugen Hoyos]

Please test this patch.

diff --git a/libavformat/asf.c b/libavformat/asf.c
index 719cae9da5..0f223d2916 100644
--- a/libavformat/asf.c
+++ b/libavformat/asf.c
@@ -105,6 +105,10 @@ const ff_asf_guid ff_asf_ext_stream_audio_stream = {
     0x9d, 0x8c, 0x17, 0x31, 0xE1, 0x03, 0x28, 0x45, 0xb5, 0x82, 0x3d, 0xf9, 0xdb, 0x22, 0xf5, 0x03
 };
 
+const ff_asf_guid ff_asf_mediatype_stream = {
+    0x83, 0xeb, 0x36, 0xe4, 0x4f, 0x52, 0xce, 0x11, 0x9f, 0x53, 0x00, 0x20, 0xaf, 0x0b, 0xa7, 0x70
+};
+
 const ff_asf_guid ff_asf_metadata_header = {
     0xea, 0xcb, 0xf8, 0xc5, 0xaf, 0x5b, 0x77, 0x48, 0x84, 0x67, 0xaa, 0x8c, 0x44, 0xfa, 0x4c, 0xca
 };
diff --git a/libavformat/asf.h b/libavformat/asf.h
index a30d3bb23b..90ce752bae 100644
--- a/libavformat/asf.h
+++ b/libavformat/asf.h
@@ -90,6 +90,7 @@ extern const ff_asf_guid ff_asf_extended_content_header;
 extern const ff_asf_guid ff_asf_simple_index_header;
 extern const ff_asf_guid ff_asf_ext_stream_embed_stream_header;
 extern const ff_asf_guid ff_asf_ext_stream_audio_stream;
+extern const ff_asf_guid ff_asf_mediatype_stream;
 extern const ff_asf_guid ff_asf_metadata_header;
 extern const ff_asf_guid ff_asf_metadata_library_header;
 extern const ff_asf_guid ff_asf_marker_header;
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index 57dc3b09b9..e1ed48d1a6 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -460,7 +460,8 @@ static int asf_read_stream_properties(AVFormatContext *s, int64_t size)
 
     if (test_for_ext_stream_audio) {
         ff_get_guid(pb, &g);
-        if (!ff_guidcmp(&g, &ff_asf_ext_stream_audio_stream)) {
+        if (   !ff_guidcmp(&g, &ff_asf_mediatype_stream)
+            || !ff_guidcmp(&g, &ff_asf_ext_stream_audio_stream)) {
             type            = AVMEDIA_TYPE_AUDIO;
             is_dvr_ms_audio = 1;
             ff_get_guid(pb, &g);

[Djfe]

Ok, done.

The patch allows ffmpeg to detect the stream as HE-AAC now,
the bitrate is wrong though (the number 1536kb/s makes it seem like it's pcm)

decoding audio-only appears to be working fine (the file is playable and has the correct length)

.\ffmpeg -i .\h1_av_175_H.asf -map 0:2 -c:a aac test.m4a

The console output is spammed massively by these two kinds of warnings though:

[aac @ 03a01f00] Queue input is backward in time
[ipod @ 04852400] Non-monotonous DTS in output stream 0:0; previous: 290926768, current: 290925744; changing to 290926769. This may result in incorrect timestamps in the output file.

copying the audio stream doesn't work at all, yet.

.\ffmpeg -i .\h1_av_175_H.asf -map 0:2 -c:a copy test.m4a
ffmpeg version N-94089-gbc3306fd5b Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7.3-win32 (GCC) 20180312
  configuration: --arch=x86 --target-os=mingw32 --cross-prefix=i686-w64-mingw32-
  libavutil      56. 29.100 / 56. 29.100
  libavcodec     58. 53.100 / 58. 53.100
  libavformat    58. 28.100 / 58. 28.100
  libavdevice    58.  7.100 / 58.  7.100
  libavfilter     7. 55.100 /  7. 55.100
  libswscale      5.  4.101 /  5.  4.101
  libswresample   3.  4.100 /  3.  4.100
Input #0, asf, from '.\h1_av_175_H.asf':
  Metadata:
    WMFSDKNeeded    : 0.0.0.0000
    WMFSDKVersion   : 10.00.00.4066
    IsVBR           : 1
  Duration: 01:41:01.03, start: 0.031000, bitrate: 1290 kb/s
    Stream #0:0(chi): Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 1920x1080, 1080 kb/s, 1k tbr, 1k tbn, 2k tbc
    Stream #0:1(chi): Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 1920x1080, 23 kb/s, 1k tbr, 1k tbn, 2k tbc
    Stream #0:2(chi): Audio: aac (HE-AAC), 48000 Hz, stereo, fltp, 1536 kb/s
    Stream #0:3(chi): Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 1920x1080, 20 kb/s, 1k tbr, 1k tbn, 2k tbc
File 'test2.m4a' already exists. Overwrite ? [y/N] y
Output #0, ipod, to 'test2.m4a':
  Metadata:
    WMFSDKNeeded    : 0.0.0.0000
    WMFSDKVersion   : 10.00.00.4066
    IsVBR           : 1
    encoder         : Lavf58.28.100
    Stream #0:0(chi): Audio: aac (HE-AAC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 1536 kb/s
Stream mapping:
  Stream #0:2 -> #0:0 (copy)
Press [q] to stop, [?] for help
size=   95047kB time=01:41:00.95 bitrate= 128.5kbits/s speed=9.07e+003x
video:0kB audio:94363kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.724151%

.\ffprobe test.m4a
ffprobe version N-94089-gbc3306fd5b Copyright (c) 2007-2019 the FFmpeg developers
  built with gcc 7.3-win32 (GCC) 20180312
  configuration: --arch=x86 --target-os=mingw32 --cross-prefix=i686-w64-mingw32-
  libavutil      56. 29.100 / 56. 29.100
  libavcodec     58. 53.100 / 58. 53.100
  libavformat    58. 28.100 / 58. 28.100
  libavdevice    58.  7.100 / 58.  7.100
  libavfilter     7. 55.100 /  7. 55.100
  libswscale      5.  4.101 /  5.  4.101
  libswresample   3.  4.100 /  3.  4.100
[aac @ 03951100] Number of bands (54) exceeds limit (47).
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test2.m4a':
  Metadata:
    major_brand     : M4A
    minor_version   : 512
    compatible_brands: isomiso2
    encoder         : Lavf58.28.100
  Duration: 01:41:01.00, start: 0.000000, bitrate: 128 kb/s
    Stream #0:0(chi): Audio: aac (Main) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 126 kb/s (default)
    Metadata:
      handler_name    : SoundHandler

the (copied) audio stream is undecodable and isn't recognized as HE-AAC (by ffprobe), (actual decoding with ffmpeg -i results in errors)