dashenc creates web-unplayable files with copied opus streams

[Gregory Beauregard]

When copying certain streams (like opus) from an input to dashenc, dashenc makes web-unplayable streams that give mp4 metadata parsing failures in Firefox/web browsers. This does not occur if the audio stream is reencoded.

ffmpeg master built in 2021/04/14
Begin by building a file to use as input for the dash encode; I'm using an mpegts to simulate a SRT-transmitted input.
ffmpeg -report -f lavfi -i testsrc=duration=60:size=1920x1080:rate=30,format=yuv420p -f lavfi -i sine=frequency=440:duration=60:sample_rate=44100:b=4 -pix_fmt yuv420p -c:v libx264 -b:v 1000k -g:v 60 -keyint_min:v 60 -sc_threshold:v 0 -profile:v high -preset veryfast -c:a libopus -b:a 128k -y -f mpegts "badmpeg.ts"
Log of this command is attached as badmpeg.log

I created the subdirectories dash/dashbad and dash/dashgood to encode with dashenc a stream where the audio is copied and reencoded respectively. The one where audio is copied will fail to playback in Firefox.

In the dash/dashbad directory, run:
ffmpeg -report -vtag "avc1" -re -i ../../badmpeg.ts -c:v:0 copy -c:a:0 copy -b:v:0 1000k -b:a:0 128k -map 0:v -map 0:a -y -f dash -seg_duration "2" -dash_segment_type mp4 -use_template 1 -use_timeline 0 -format_options "movflags=+cmaf" -adaptation_sets "id=0,streams=v id=1,streams=a" "stream.mpd"
In the dash/dashgood directory, run:
ffmpeg -report -vtag "avc1" -re -i ../../badmpeg.ts -c:v:0 copy -c:a:0 libopus -b:a:0 64k -b:v:0 1000k -map 0:v -map 0:a -y -f dash -seg_duration "2" -dash_segment_type mp4 -use_template 1 -use_timeline 0 -format_options "movflags=+cmaf" -adaptation_sets "id=0,streams=v id=1,streams=a" "stream.mpd"

The only difference for these two dashenc runs is that in the "dashgood" one the opus input is re-encoded. The logs of these two runs are attached as dashgood.log and dashbad.log.

If you try to play the stream.mpd in the dashgood run where the opus audio was reencoded, it will play fine with dash.js in a browser. If you try to play the dashbad run it will fail with an mp4 parsing error e.g. the following in Firefox 87.0

MEDIA_ERR_SRC_NOT_SUPPORTED (NS_ERROR_DOM_MEDIA_METADATA_ERR (0x806e0006) - virtual RefPtr<MP4Demuxer::InitPromise> mozilla::MP4Demuxer::Init(): Parse MP4 metadata failed)

I don't have any issues with ffplay or mpv on either file.

I am hosting the streams on my website so you can reproduce the failure with the dash.js reference player at ​https://reference.dashif.org/dash.js/v3.2.2/samples/dash-if-reference-player/index.html.

You may try playing ​https://stream.gably.net/images/dash/dashgood/stream.mpd and ​https://stream.gably.net/images/dash/dashbad/stream.mpd with the reference player for the working (transcoded audio) and nonworking (copied audio) streams respectively.

[James]

Can you apply this patch and re-run the command line to create dash/dashbad, to see if it fixes it?

diff --git a/libavformat/mpegts.c b/libavformat/mpegts.c
index 6e0d9d7496..cc4a9e6f62 100644
--- a/libavformat/mpegts.c
+++ b/libavformat/mpegts.c
@@ -2018,7 +2018,6 @@ int ff_parse_mpeg2_descriptor(AVFormatContext *fc, AVStream *st, int stream_type
                 if (!st->codecpar->extradata)
                     return AVERROR(ENOMEM);

-                st->codecpar->extradata_size = sizeof(opus_default_extradata);
                 memcpy(st->codecpar->extradata, opus_default_extradata, sizeof(opus_default_extradata));

                 channel_config_code = get8(pp, desc_end);
@@ -2030,8 +2029,10 @@ int ff_parse_mpeg2_descriptor(AVFormatContext *fc, AVStream *st, int stream_type
                     st->codecpar->extradata[19] = opus_stream_cnt[channel_config_code];
                     st->codecpar->extradata[20] = opus_coupled_stream_cnt[channel_config_code];
                     memcpy(&st->codecpar->extradata[21], opus_channel_map[channels - 1], channels);
+                    st->codecpar->extradata_size = 19 + (st->codecpar->extradata[18] ? 2 + channels : 0);
                 } else {
                     avpriv_request_sample(fc, "Opus in MPEG-TS - channel_config_code > 0x8");
+                    st->codecpar->extradata_size = sizeof(opus_default_extradata);
                 }
                 st->need_parsing = AVSTREAM_PARSE_FULL;
                 st->internal->need_context_update = 1;

[Gregory Beauregard]

Replying to jamrial:

Can you apply this patch and re-run the command line to create dash/dashbad, to see if it fixes it?

diff --git a/libavformat/mpegts.c b/libavformat/mpegts.c
index 6e0d9d7496..cc4a9e6f62 100644
--- a/libavformat/mpegts.c
+++ b/libavformat/mpegts.c
@@ -2018,7 +2018,6 @@ int ff_parse_mpeg2_descriptor(AVFormatContext *fc, AVStream *st, int stream_type
                 if (!st->codecpar->extradata)
                     return AVERROR(ENOMEM);

-                st->codecpar->extradata_size = sizeof(opus_default_extradata);
                 memcpy(st->codecpar->extradata, opus_default_extradata, sizeof(opus_default_extradata));

                 channel_config_code = get8(pp, desc_end);
@@ -2030,8 +2029,10 @@ int ff_parse_mpeg2_descriptor(AVFormatContext *fc, AVStream *st, int stream_type
                     st->codecpar->extradata[19] = opus_stream_cnt[channel_config_code];
                     st->codecpar->extradata[20] = opus_coupled_stream_cnt[channel_config_code];
                     memcpy(&st->codecpar->extradata[21], opus_channel_map[channels - 1], channels);
+                    st->codecpar->extradata_size = 19 + (st->codecpar->extradata[18] ? 2 + channels : 0);
                 } else {
                     avpriv_request_sample(fc, "Opus in MPEG-TS - channel_config_code > 0x8");
+                    st->codecpar->extradata_size = sizeof(opus_default_extradata);
                 }
                 st->need_parsing = AVSTREAM_PARSE_FULL;
                 st->internal->need_context_update = 1;

This results in a working stream with the dashbad command. See ​https://stream.gably.net/images/dash/dashtest/stream.mpd for a test stream with the copied opus and your patch ("dashtest").

[James]

Ok, patch sent to the ml.

[Gregory Beauregard]

Replying to jamrial:

Ok, patch sent to the ml.

While the patch fixed the playback with Firefox, Chrome still gives an error with dashtest:

MEDIA_ERR_SRC_NOT_SUPPORTED (CHUNK_DEMUXER_ERROR_APPEND_FAILED: Failure parsing MP4: Opus AudioSampleEntry sample rate mismatches OpusSpecificBox STREAMINFO channel count)

I noticed you posted a different patch to the mailing list as well; I'll try that when I get a chance.

EDIT: the second ML patch doesn't fix the Chrome playback. Here's a test file with only the second patch applied "dashtest2" ​http://ffmpeg.org/pipermail/ffmpeg-devel/2021-April/279062.html: ​https://stream.gably.net/images/dash/dashtest2/stream.mpd (this is still fixed on Firefox for me)

[James]

Fixed by bc010612318ea0bbd31d4b92cbb90ccfbed09631 and c1fd4033dca192d709ff01203aa833256df5b70f.