dash muxer does not specify full codec string parameters for AV1

[Brion Vibber]

Summary of the bug:

When using the dash muxer to produce DASH output containing an AV1 video stream, the codec string is listed only as "av01" without the full parameters, such as "av01.0.04M.08".

This results in failure to play in Chrome, where MediaSource.isTypeSupported('video/mp4; codecs="av01"') without those parameters returns false.

The parameters should be extracted from the extradata per ​https://aomediacodec.github.io/av1-isobmff/#codecsparam

How to reproduce:

% $ ffmpeg -t 1 -i crowd_run_1080p50.y4m -s 640x360 -strict -2 -c:v libaom-av1 -cpu-used 8 -tile-columns 2 -g 50 -keyint_min 50 -f dash foo.mpd
ffmpeg version 4.1.4 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.1 (clang-1001.0.46.4)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.4_1 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags='-I/Library/Java/JavaVirtualMachines/adoptopenjdk-12.0.1.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/adoptopenjdk-12.0.1.jdk/Contents/Home/include/darwin' --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack --disable-indev=jack --enable-libaom --enable-libsoxr
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, yuv4mpegpipe, from 'crowd_run_1080p50.y4m':
  Duration: 00:00:10.00, start: 0.000000, bitrate: 1244162 kb/s
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p(progressive), 1920x1080, SAR 1:1 DAR 16:9, 50 fps, 50 tbr, 50 tbn, 50 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> av1 (libaom-av1))
Press [q] to stop, [?] for help
[libaom-av1 @ 0x7fb83f002a00] 1.0.0
[dash @ 0x7fb83f012a00] Opening 'init-stream0.m4s' for writing
Output #0, dash, to 'foo.mpd':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0: Video: av1 (libaom-av1), yuv420p, 640x360 [SAR 1:1 DAR 16:9], q=-1--1, 200 kb/s, 50 fps, 12800 tbn, 50 tbc
    Metadata:
      encoder         : Lavc58.35.100 libaom-av1
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
[dash @ 0x7fb83f012a00] Opening 'chunk-stream0-00001.m4s.tmp' for writing
[dash @ 0x7fb83f012a00] Opening 'foo.mpd.tmp' for writingate=N/A speed=0.0205x    
frame=   50 fps=1.0 q=-0.0 Lsize=N/A time=00:00:00.98 bitrate=N/A speed=0.02x    
video:88kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

% $ cat foo.mpd

<?xml version="1.0" encoding="utf-8"?>
<MPD xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns="urn:mpeg:dash:schema:mpd:2011"
	xmlns:xlink="http://www.w3.org/1999/xlink"
	xsi:schemaLocation="urn:mpeg:DASH:schema:MPD:2011 http://standards.iso.org/ittf/PubliclyAvailableStandards/MPEG-DASH_schema_files/DASH-MPD.xsd"
	profiles="urn:mpeg:dash:profile:isoff-live:2011"
	type="static"
	mediaPresentationDuration="PT1.0S"
	minBufferTime="PT2.0S">
	<ProgramInformation>
	</ProgramInformation>
	<Period id="0" start="PT0.0S">
		<AdaptationSet id="0" contentType="video" segmentAlignment="true" bitstreamSwitching="true">
			<Representation id="0" mimeType="video/mp4" codecs="av01" bandwidth="200000" width="640" height="360" frameRate="50/1">
				<SegmentTemplate timescale="12800" initialization="init-stream$RepresentationID$.m4s" media="chunk-stream$RepresentationID$-$Number%05d$.m4s" startNumber="1">
					<SegmentTimeline>
						<S t="0" d="12800" />
					</SegmentTimeline>
				</SegmentTemplate>
			</Representation>
		</AdaptationSet>
	</Period>
</MPD>

[Carl Eugen Hoyos]

Is this issue reproducible with current FFmpeg git head?

[James]

The dash muxer is effectively not setting the codec string with the minimum required values beyond the sample entry 4CC.

Unfortunately, even if we were to make the muxer look at the stream's extradata to get the required information, the libaom encoder is currently not propagating said extradata to begin with because of a known bug[1] in their API that hasn't been fixed in almost a year now. It would work only in remuxing scenarios until that's solved.

[1] ​https://bugs.chromium.org/p/aomedia/issues/detail?id=2208

[Brion Vibber]

Ah, thanks for the pointer! I'll take a look at that bug over on aom.

[James]

Turns out the dash muxer can use extradata propagated late during the muxing process. It just wasn't looking at the correct place for it.

Fixed that in ce6a98e8306105b07bd7653f4f10c23fd75914ee, and the codec string for AV1 streams in 1cf2f040e34bbfedde60ff3d91b2f7b770aca85b.