Opened 7 years ago

Closed 6 years ago

#6610 closed enhancement (fixed)

Multiple video AdaptionSets with dash

Reported by: Paul Owned by:
Priority: wish Component: avformat
Version: git-master Keywords: dash
Cc: Daemon404 Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

I am trying to create a dash manifest which includes multiple video AdaptationSets (specifically DASH, not webm_dash). However, it seems all output streams of AVMEDIA_TYPE_VIDEO are automatically grouped within a single AdaptationSet, and categorized as additional Representations within the dash manifest.

While webm_dash_manifest supports the adaptationSets parameter, reviewing dashenc.c, it does not seem there is currently a way to include multiple video AdaptationSets in a dash manifest.

Since Representations are intended to only differ in quality level (resolution / bitrate), and AdaptationSets are intended for different content, this is a limitation because there is no way to include multiple videos of different content (such as different camera angles) in a dash stream.

How to reproduce:

% ffmpeg -i in1.mov -i in2.mov -map 0:v -map 0:a -map 1:v -f dash dashtest/Manifest.mpd
ffmpeg version N-87027-gd826951 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.1.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-zlib
  libavutil      55. 74.100 / 55. 74.100
  libavcodec     57.103.100 / 57.103.100
  libavformat    57. 76.100 / 57. 76.100
  libavdevice    57.  7.100 / 57.  7.100
  libavfilter     6.100.100 /  6.100.100
  libswscale      4.  7.102 /  4.  7.102
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'in1.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 0
    compatible_brands: qt
    creation_time   : 2017-08-06T14:07:37.000000Z
    com.apple.quicktime.location.ISO6709: +38.9344-104.8211+1934.449/
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone 6s Plus
    com.apple.quicktime.software: 10.3.3
    com.apple.quicktime.creationdate: 2017-08-06T08:04:37-0600
  Duration: 00:00:04.49, start: 0.000000, bitrate: 2338 kb/s
    Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 80 kb/s (default)
    Metadata:
      creation_time   : 2017-08-06T14:07:37.000000Z
      handler_name    : Core Media Data Handler
    Stream #0:1(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720, 2234 kb/s, 30 fps, 30 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      creation_time   : 2017-08-06T14:07:37.000000Z
      handler_name    : Core Media Data Handler
      encoder         : H.264
    Stream #0:2(und): Data: none (mebx / 0x7862656D), 4 kb/s (default)
    Metadata:
      creation_time   : 2017-08-06T14:07:37.000000Z
      handler_name    : Core Media Data Handler
    Stream #0:3(und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
    Metadata:
      creation_time   : 2017-08-06T14:07:37.000000Z
      handler_name    : Core Media Data Handler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'in2.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 0
    compatible_brands: qt
    creation_time   : 2017-08-06T14:07:37.000000Z
    com.apple.quicktime.location.ISO6709: +38.9344-104.8211+1934.449/
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone 6s Plus
    com.apple.quicktime.software: 10.3.3
    com.apple.quicktime.creationdate: 2017-08-06T08:04:37-0600
  Duration: 00:00:04.49, start: 0.000000, bitrate: 2338 kb/s
    Stream #1:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 80 kb/s (default)
    Metadata:
      creation_time   : 2017-08-06T14:07:37.000000Z
      handler_name    : Core Media Data Handler
    Stream #1:1(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720, 2234 kb/s, 30 fps, 30 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      creation_time   : 2017-08-06T14:07:37.000000Z
      handler_name    : Core Media Data Handler
      encoder         : H.264
    Stream #1:2(und): Data: none (mebx / 0x7862656D), 4 kb/s (default)
    Metadata:
      creation_time   : 2017-08-06T14:07:37.000000Z
      handler_name    : Core Media Data Handler
    Stream #1:3(und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
    Metadata:
      creation_time   : 2017-08-06T14:07:37.000000Z
      handler_name    : Core Media Data Handler
Stream mapping:
  Stream #0:1 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:0 -> #0:1 (aac (native) -> aac (native))
  Stream #1:1 -> #0:2 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0000000002692300] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000000002692300] profile High, level 3.1
[libx264 @ 0000000002692300] 264 - core 152 r2851 ba24899 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=22 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[libx264 @ 0000000002695320] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000000002695320] profile High, level 3.1
[libx264 @ 0000000002695320] 264 - core 152 r2851 ba24899 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=22 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[dash @ 0000000002696940] No bit rate set for stream 0
[dash @ 0000000002696940] No bit rate set for stream 2
[dash @ 0000000002696940] Opening 'dashtest/init-stream0.m4s' for writing
[dash @ 0000000002696940] Opening 'dashtest/init-stream1.m4s' for writing
[dash @ 0000000002696940] Opening 'dashtest/init-stream2.m4s' for writing
Output #0, dash, to 'dashtest/Manifest.mpd':
  Metadata:
    major_brand     : qt
    minor_version   : 0
    compatible_brands: qt
    com.apple.quicktime.creationdate: 2017-08-06T08:04:37-0600
    com.apple.quicktime.location.ISO6709: +38.9344-104.8211+1934.449/
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone 6s Plus
    com.apple.quicktime.software: 10.3.3
    encoder         : Lavf57.76.100
    Stream #0:0(und): Video: h264 (libx264), yuv420p(progressive), 1280x720, q=-1--1, 30 fps, 15360 tbn, 30 tbc (default)
    Metadata:
      creation_time   : 2017-08-06T14:07:37.000000Z
      handler_name    : Core Media Data Handler
      encoder         : Lavc57.103.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1(und): Audio: aac (LC), 44100 Hz, mono, fltp, 69 kb/s (default)
    Metadata:
      creation_time   : 2017-08-06T14:07:37.000000Z
      handler_name    : Core Media Data Handler
      encoder         : Lavc57.103.100 aac
    Stream #0:2(und): Video: h264 (libx264), yuv420p, 1280x720, q=-1--1, 30 fps, 15360 tbn, 30 tbc (default)
    Metadata:
      creation_time   : 2017-08-06T14:07:37.000000Z
      handler_name    : Core Media Data Handler
      encoder         : Lavc57.103.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
[dash @ 0000000002696940] Opening 'dashtest/Manifest.mpd.tmp' for writinged=6.59x
[dash @ 0000000002696940] No bit rate set for stream 004.45 bitrate=N/A speed=2.96x
[dash @ 0000000002696940] No bit rate set for stream 2
[dash @ 0000000002696940] Opening 'dashtest/chunk-stream0-00001.m4s.tmp' for writing
[dash @ 0000000002696940] Opening 'dashtest/chunk-stream1-00001.m4s.tmp' for writing
[dash @ 0000000002696940] Opening 'dashtest/chunk-stream2-00001.m4s.tmp' for writing
[dash @ 0000000002696940] Opening 'dashtest/Manifest.mpd.tmp' for writing
frame=  135 fps= 59 q=-1.0 Lq=-1.0 size=N/A time=00:00:04.50 bitrate=N/A speed=1.98x
video:1866kB audio:39kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0000000002692300] frame I:2     Avg QP:21.77  size: 29000
[libx264 @ 0000000002692300] frame P:46    Avg QP:22.70  size: 12893
[libx264 @ 0000000002692300] frame B:87    Avg QP:24.85  size:  3489
[libx264 @ 0000000002692300] consecutive B-frames:  2.2% 35.6%  0.0% 62.2%
[libx264 @ 0000000002692300] mb I  I16..4: 26.3% 53.2% 20.5%
[libx264 @ 0000000002692300] mb P  I16..4:  8.8%  9.2%  1.5%  P16..4: 45.8%  8.7%  3.6%  0.0%  0.0%    skip:22.5%
[libx264 @ 0000000002692300] mb B  I16..4:  1.0%  0.8%  0.1%  B16..8: 33.8%  2.5%  0.2%  direct: 2.6%  skip:59.1%  L0:43.6% L1:52.8% BI: 3.6%
[libx264 @ 0000000002692300] 8x8 transform intra:47.5% inter:80.6%
[libx264 @ 0000000002692300] coded y,uvDC,uvAC intra: 26.1% 63.7% 17.0% inter: 7.3% 20.2% 1.1%
[libx264 @ 0000000002692300] i16 v,h,dc,p: 39% 34% 13% 14%
[libx264 @ 0000000002692300] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 30% 22% 37%  2%  2%  2%  2%  2%  2%
[libx264 @ 0000000002692300] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 38% 24% 12%  4%  5%  5%  4%  4%  4%
[libx264 @ 0000000002692300] i8c dc,h,v,p: 40% 25% 27%  8%
[libx264 @ 0000000002692300] Weighted P-Frames: Y:2.2% UV:0.0%
[libx264 @ 0000000002692300] ref P L0: 72.0%  8.3% 14.2%  5.5%
[libx264 @ 0000000002692300] ref B L0: 92.2%  6.3%  1.5%
[libx264 @ 0000000002692300] ref B L1: 97.8%  2.2%
[libx264 @ 0000000002692300] kb/s:1697.10
[aac @ 0000000002693b20] Qavg: 966.864
[libx264 @ 0000000002695320] frame I:2     Avg QP:21.77  size: 29000
[libx264 @ 0000000002695320] frame P:46    Avg QP:22.70  size: 12893
[libx264 @ 0000000002695320] frame B:87    Avg QP:24.85  size:  3489
[libx264 @ 0000000002695320] consecutive B-frames:  2.2% 35.6%  0.0% 62.2%
[libx264 @ 0000000002695320] mb I  I16..4: 26.3% 53.2% 20.5%
[libx264 @ 0000000002695320] mb P  I16..4:  8.8%  9.2%  1.5%  P16..4: 45.8%  8.7%  3.6%  0.0%  0.0%    skip:22.5%
[libx264 @ 0000000002695320] mb B  I16..4:  1.0%  0.8%  0.1%  B16..8: 33.8%  2.5%  0.2%  direct: 2.6%  skip:59.1%  L0:43.6% L1:52.8% BI: 3.6%
[libx264 @ 0000000002695320] 8x8 transform intra:47.5% inter:80.6%
[libx264 @ 0000000002695320] coded y,uvDC,uvAC intra: 26.1% 63.7% 17.0% inter: 7.3% 20.2% 1.1%
[libx264 @ 0000000002695320] i16 v,h,dc,p: 39% 34% 13% 14%
[libx264 @ 0000000002695320] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 30% 22% 37%  2%  2%  2%  2%  2%  2%
[libx264 @ 0000000002695320] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 38% 24% 12%  4%  5%  5%  4%  4%  4%
[libx264 @ 0000000002695320] i8c dc,h,v,p: 40% 25% 27%  8%
[libx264 @ 0000000002695320] Weighted P-Frames: Y:2.2% UV:0.0%
[libx264 @ 0000000002695320] ref P L0: 72.0%  8.3% 14.2%  5.5%
[libx264 @ 0000000002695320] ref B L0: 92.2%  6.3%  1.5%
[libx264 @ 0000000002695320] ref B L1: 97.8%  2.2%
[libx264 @ 0000000002695320] kb/s:1697.10

Produces the following manifest:

<?xml version="1.0" encoding="utf-8"?>
<MPD xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns="urn:mpeg:dash:schema:mpd:2011"
	xmlns:xlink="http://www.w3.org/1999/xlink"
	xsi:schemaLocation="urn:mpeg:DASH:schema:MPD:2011 http://standards.iso.org/ittf/PubliclyAvailableStandards/MPEG-DASH_schema_files/DASH-MPD.xsd"
	profiles="urn:mpeg:dash:profile:isoff-live:2011"
	type="static"
	mediaPresentationDuration="PT4.5S"
	minBufferTime="PT9.0S">
	<ProgramInformation>
	</ProgramInformation>
	<Period id="0" start="PT0.0S">
		<AdaptationSet id="0" contentType="video" segmentAlignment="true" bitstreamSwitching="true" frameRate="30/1">
			<Representation id="0" mimeType="video/mp4" codecs="avc1.64001f" width="1280" height="720" frameRate="30/1">
				<SegmentTemplate timescale="15360" initialization="init-stream$RepresentationID$.m4s" media="chunk-stream$RepresentationID$-$Number%05d$.m4s" startNumber="1">
					<SegmentTimeline>
						<S t="0" d="69120" />
					</SegmentTimeline>
				</SegmentTemplate>
			</Representation>
			<Representation id="2" mimeType="video/mp4" codecs="avc1.64001f" width="1280" height="720" frameRate="30/1">
				<SegmentTemplate timescale="15360" initialization="init-stream$RepresentationID$.m4s" media="chunk-stream$RepresentationID$-$Number%05d$.m4s" startNumber="1">
					<SegmentTimeline>
						<S t="0" d="69120" />
					</SegmentTimeline>
				</SegmentTemplate>
			</Representation>
		</AdaptationSet>
		<AdaptationSet id="1" contentType="audio" segmentAlignment="true" bitstreamSwitching="true">
			<Representation id="1" mimeType="audio/mp4" codecs="mp4a.40.2" bandwidth="69000" audioSamplingRate="44100">
				<AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="1" />
				<SegmentTemplate timescale="44100" initialization="init-stream$RepresentationID$.m4s" media="chunk-stream$RepresentationID$-$Number%05d$.m4s" startNumber="1">
					<SegmentTimeline>
						<S t="0" d="198656" />
					</SegmentTimeline>
				</SegmentTemplate>
			</Representation>
		</AdaptationSet>
	</Period>
</MPD>

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

Change History (2)

comment:1 by Carl Eugen Hoyos, 6 years ago

Component: undetermined → avformat
Keywords: manifest adaptationSet representation dashenc.c removed
Priority: normal → wish
Type: defect → enhancement

Is this still reproducible or was this implemented by (Peter Große's) committed - or merged - patches?

in reply to:  1 comment:2 by Paul, 6 years ago

Resolution: → fixed
Status: new → closed

Replying to cehoyos:

Is this still reproducible or was this implemented by (Peter Große's) committed - or merged - patches?

This is no longer reproducible on the newest version.. appears to have been solved. Thank you!

Note: See TracTickets for help on using tickets.