Opened 5 years ago

Closed 5 years ago

#8049 closed defect (fixed)

dash muxer does not specify full codec string parameters for AV1

Reported by: Brion Vibber Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: av1 dash
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: yes

Description

Summary of the bug:

When using the dash muxer to produce DASH output containing an AV1 video stream, the codec string is listed only as "av01" without the full parameters, such as "av01.0.04M.08".

This results in failure to play in Chrome, where MediaSource.isTypeSupported('video/mp4; codecs="av01"') without those parameters returns false.

The parameters should be extracted from the extradata per https://aomediacodec.github.io/av1-isobmff/#codecsparam

How to reproduce:

% $ ffmpeg -t 1 -i crowd_run_1080p50.y4m -s 640x360 -strict -2 -c:v libaom-av1 -cpu-used 8 -tile-columns 2 -g 50 -keyint_min 50 -f dash foo.mpd
ffmpeg version 4.1.4 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.1 (clang-1001.0.46.4)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.4_1 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags='-I/Library/Java/JavaVirtualMachines/adoptopenjdk-12.0.1.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/adoptopenjdk-12.0.1.jdk/Contents/Home/include/darwin' --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack --disable-indev=jack --enable-libaom --enable-libsoxr
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, yuv4mpegpipe, from 'crowd_run_1080p50.y4m':
  Duration: 00:00:10.00, start: 0.000000, bitrate: 1244162 kb/s
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p(progressive), 1920x1080, SAR 1:1 DAR 16:9, 50 fps, 50 tbr, 50 tbn, 50 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> av1 (libaom-av1))
Press [q] to stop, [?] for help
[libaom-av1 @ 0x7fb83f002a00] 1.0.0
[dash @ 0x7fb83f012a00] Opening 'init-stream0.m4s' for writing
Output #0, dash, to 'foo.mpd':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0: Video: av1 (libaom-av1), yuv420p, 640x360 [SAR 1:1 DAR 16:9], q=-1--1, 200 kb/s, 50 fps, 12800 tbn, 50 tbc
    Metadata:
      encoder         : Lavc58.35.100 libaom-av1
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
[dash @ 0x7fb83f012a00] Opening 'chunk-stream0-00001.m4s.tmp' for writing
[dash @ 0x7fb83f012a00] Opening 'foo.mpd.tmp' for writingate=N/A speed=0.0205x    
frame=   50 fps=1.0 q=-0.0 Lsize=N/A time=00:00:00.98 bitrate=N/A speed=0.02x    
video:88kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

% $ cat foo.mpd

<?xml version="1.0" encoding="utf-8"?>
<MPD xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns="urn:mpeg:dash:schema:mpd:2011"
	xmlns:xlink="http://www.w3.org/1999/xlink"
	xsi:schemaLocation="urn:mpeg:DASH:schema:MPD:2011 http://standards.iso.org/ittf/PubliclyAvailableStandards/MPEG-DASH_schema_files/DASH-MPD.xsd"
	profiles="urn:mpeg:dash:profile:isoff-live:2011"
	type="static"
	mediaPresentationDuration="PT1.0S"
	minBufferTime="PT2.0S">
	<ProgramInformation>
	</ProgramInformation>
	<Period id="0" start="PT0.0S">
		<AdaptationSet id="0" contentType="video" segmentAlignment="true" bitstreamSwitching="true">
			<Representation id="0" mimeType="video/mp4" codecs="av01" bandwidth="200000" width="640" height="360" frameRate="50/1">
				<SegmentTemplate timescale="12800" initialization="init-stream$RepresentationID$.m4s" media="chunk-stream$RepresentationID$-$Number%05d$.m4s" startNumber="1">
					<SegmentTimeline>
						<S t="0" d="12800" />
					</SegmentTimeline>
				</SegmentTemplate>
			</Representation>
		</AdaptationSet>
	</Period>
</MPD>

Change History (4)

comment:1 by Carl Eugen Hoyos, 5 years ago

Keywords: av1,dash → av1 dash
Priority: normalminor

Is this issue reproducible with current FFmpeg git head?

comment:2 by James, 5 years ago

Analyzed by developer: set
Component: undeterminedavformat
Priority: minornormal
Reproduced by developer: set
Status: newopen
Version: unspecifiedgit-master

The dash muxer is effectively not setting the codec string with the minimum required values beyond the sample entry 4CC.

Unfortunately, even if we were to make the muxer look at the stream's extradata to get the required information, the libaom encoder is currently not propagating said extradata to begin with because of a known bug[1] in their API that hasn't been fixed in almost a year now. It would work only in remuxing scenarios until that's solved.

[1] https://bugs.chromium.org/p/aomedia/issues/detail?id=2208

comment:3 by Brion Vibber, 5 years ago

Ah, thanks for the pointer! I'll take a look at that bug over on aom.

comment:4 by James, 5 years ago

Resolution: fixed
Status: openclosed

Turns out the dash muxer can use extradata propagated late during the muxing process. It just wasn't looking at the correct place for it.

Fixed that in ce6a98e8306105b07bd7653f4f10c23fd75914ee, and the codec string for AV1 streams in 1cf2f040e34bbfedde60ff3d91b2f7b770aca85b.

Note: See TracTickets for help on using tickets.