Opened 2 years ago

Last modified 8 weeks ago

#4178 open defect

Opus audio in MKV container

Reported by: agressiv Owned by: vigneshvg
Priority: important Component: avformat
Version: git-master Keywords: mkv opus regression
Cc: moritz@bunkus.org, michael Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: yes

Description

Summary of the bug:

Opus Audio in MKV/WEBM/MKA containers is has crackling and lots of clicks/artifacts when played in ffplay, mplayer or using shared libraries (e.g. Kodi)

How to reproduce:

Mux Opus Audio into MKV.  Play it with ffplay, mplayer
or anything leveraging ffmpeg shared libraries.

MPC-HC has zero problems playing these files at all. I've taken the first minute of a bad MKA (for sizes' sake) and have attached it. If you extract the Opus, ffplay will play it fine.

Attachments (9)

Star Trek (2009).mka (2.2 MB) - added by agressiv 2 years ago.
1 minute of an MKA container with an Opus sound file
st6.mka (712.5 KB) - added by agressiv 2 years ago.
st6.opus (707.3 KB) - added by agressiv 2 years ago.
st7.mka (725.4 KB) - added by agressiv 2 years ago.
st7.opus (707.3 KB) - added by agressiv 2 years ago.
st9.mka (712.5 KB) - added by agressiv 2 years ago.
st9.opus (707.3 KB) - added by agressiv 2 years ago.
st10.mka (710.4 KB) - added by agressiv 2 years ago.
st10.opus (706.7 KB) - added by agressiv 2 years ago.

Change History (26)

Changed 2 years ago by agressiv

1 minute of an MKA container with an Opus sound file

comment:1 Changed 2 years ago by cehoyos

  • Keywords regression added; libopus matroska crackling removed
  • Priority changed from normal to important
  • Reproduced by developer set
  • Status changed from new to open
  • Version changed from 2.5 to git-master

Regression since 7b0a839b

For future tickets: Please always test current FFmpeg git head and please provide complete, uncut output of the failing command to make the tickets valid.

$ ffmpeg -i Star\ Trek\ \(2009\).mka out.wav
ffmpeg version N-68356-gd43d5c5 Copyright (c) 2000-2014 the FFmpeg developers
  built on Dec 10 2014 10:07:42 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      54. 15.100 / 54. 15.100
  libavcodec     56. 14.100 / 56. 14.100
  libavformat    56. 15.103 / 56. 15.103
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  2.103 /  5.  2.103
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, matroska,webm, from 'Star Trek (2009).mka':
  Metadata:
    title           : StarTrek2009
    ENCODER         : Lavf56.11.100
  Duration: 00:01:00.00, start: 0.007000, bitrate: 300 kb/s
    Chapter #0:0: start 0.007000, end 60.000000
    Metadata:
      title           : Chapter 01
    Stream #0:0(eng): Audio: opus, 48000 Hz, 5.1, fltp (default)
    Metadata:
      LANGUAGE        : eng
      BPS             : 253416
      BPS-eng         : 253416
      DURATION        : 02:06:50.260000000
      DURATION-eng    : 02:06:50.260000000
      NUMBER_OF_FRAMES: 380514
      NUMBER_OF_FRAMES-eng: 380514
      NUMBER_OF_BYTES : 241070856
      NUMBER_OF_BYTES-eng: 241070856
      _STATISTICS_WRITING_APP: mkvmerge v7.3.0 ('Nouages') 64bit built on Oct 22 2014 18:53:34
      _STATISTICS_WRITING_APP-eng: mkvmerge v7.3.0 ('Nouages') 64bit built on Oct 22 2014 18:53:34
      _STATISTICS_WRITING_DATE_UTC: 2014-11-23 21:24:11
      _STATISTICS_WRITING_DATE_UTC-eng: 2014-11-23 21:24:11
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Output #0, wav, to 'out.wav':
  Metadata:
    INAM            : StarTrek2009
    ISFT            : Lavf56.15.103
    Chapter #0:0: start 0.014000, end 60.007000
    Metadata:
      title           : Chapter 01
    Stream #0:0(eng): Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, 5.1, s16, 4608 kb/s (default)
    Metadata:
      LANGUAGE        : eng
      BPS             : 253416
      BPS-eng         : 253416
      DURATION        : 02:06:50.260000000
      DURATION-eng    : 02:06:50.260000000
      NUMBER_OF_FRAMES: 380514
      NUMBER_OF_FRAMES-eng: 380514
      NUMBER_OF_BYTES : 241070856
      NUMBER_OF_BYTES-eng: 241070856
      _STATISTICS_WRITING_APP: mkvmerge v7.3.0 ('Nouages') 64bit built on Oct 22 2014 18:53:34
      _STATISTICS_WRITING_APP-eng: mkvmerge v7.3.0 ('Nouages') 64bit built on Oct 22 2014 18:53:34
      _STATISTICS_WRITING_DATE_UTC: 2014-11-23 21:24:11
      _STATISTICS_WRITING_DATE_UTC-eng: 2014-11-23 21:24:11
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      encoder         : Lavc56.14.100 pcm_s16le
Stream mapping:
  Stream #0:0 -> #0:0 (opus (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
size=   21676kB time=00:01:00.00 bitrate=2959.5kbits/s
video:0kB audio:21676kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000559%

Also reproducible with the libopus decoder, the following produces a playable file:

$ ffmpeg -i Star\ Trek\ \(2009\).mka -acodec copy out.opus

comment:2 Changed 2 years ago by vigneshvg

Author of change 7b0a839b (the regression) here.

I looked into the attached sample file, it seems like it has DiscardPadding? elements in frames where it shouldn't. DiscardPadding? is supposed to be present only in the very last frame of the file. Whereas the attached sample file has DiscardPadding? element scattered all over the file.

Could you please tell if this file was created using ffmpeg? If so, can you please tell the exact command line you used to encode this file?

If this file was created by some other tool, then the file is invalid and the crackling artifacts are intended behavior as per the DiscardPadding? element.

comment:3 Changed 2 years ago by vigneshvg

  • Analyzed by developer set
  • Owner set to vigneshvg

comment:4 Changed 2 years ago by agressiv

The source mkv was about 4gb, I used ffmpeg to trim it to an mka (taking out the video stream and limiting it to the first 20 seconds) in order to facilitate uploading as part of this ticket.

However, the source mkv file (which plays fine in MPC-HC) was created with mkvmerge.exe (however the audio was encoded with ffmpeg).

So, it seems that ffplay (and the DLL libraries) don't like mkv files created by mkvmerge with Opus in the container, if I am getting that right. (And I tried at least 5-6 versions of mkvmerge at the time) MPC-HC is able to "discard" the "DiscardPadding??" elements, ironically.

When I get back in town this weekend I'll completely demux the source file, and use ffmpeg as the muxer instead of mkvmerge.

Could mkvmerge really be non-compliant, or would you think it's just a bug with how mkvmerge handles Opus Muxes? I'm not a developer but I can certainly open a bug with mkvtoolnix if need be. Any added details you can provide would be helpful.

comment:5 Changed 2 years ago by heleppkes

I'm the author of the MKV demuxer in MPC-HC, and Discard Padding is indeed currently ignored, but mostly by accident and not by design (or rather, due to a missing feature, the DirectShow filter concept used by MPC-HC doesn't make communicating side-data from demux to decoder quite trivial).

If the sample contains DiscardPadding? in frames that shouldn't have them, then I would argue that the file is muxed wrong. mkvmerge is not fool proof, especially when it comes to new things like Opus, so it might be worth inquiring with its author (or checking the code).

comment:6 Changed 2 years ago by heleppkes

To help analyse the source, where did the Opus audio comes from originally before muxing it to MKV? Was it encoded into .ogg/.opus? It might as well be a bug in the Ogg/Opus? reader in mkvmerge, or even a bug in the .opus muxer in ffmpeg (if thats what encoded the audio).

Last edited 2 years ago by heleppkes (previous) (diff)

comment:7 Changed 2 years ago by agressiv

The Opus file was created by ffmpeg from a variety of sources, usually DTS-HD or TrueHD. I also have some coming from opusenc piped from eac3to to as well, but I haven't used that in a while - however I could do some more experimentation there as well.

comment:8 Changed 2 years ago by mbunkus

The MKVToolNix' author here. For each Opus packet read from an Ogg file mkvmerge decodes the Opus TOC and calculates the expected decoded duration (and the expected end timestamp as the sum of the durations of all prior Opus packets). This duration is compared with the Ogg page's granulepos which signals when the containing packets are supposed to _end_. If there is a difference between what the Opus TOC and the Ogg page's granulepos say then this difference is written as a DiscardPadding element.

I've done some quick tests. For files created by opusenc there's exactly one Ogg page/Opus packet for which this is the case: the very last one in the file. The same is true for files created with ffmpeg 2.5 (from the Arch Linux repositories). The resulting Matroska files contain exactly one DiscardPadding element.

I've also given today's ffmpeg git a try (revision a30fd828abf4830d15a1bd7935d08477961f6628). That version does not show any irregularities either; mkvmerge only finds a single Ogg packet with such a difference.

Therefore I'm interested how the Ogg/Opus files the ticket's submitter has used were created. The amount of DiscardPadding elements in the sample file attached to this ticket suggests that pretty much all Ogg pages have granulepos values that differ from what the Opus TOC suggests.

I'd also like to get my hands on such an Ogg/Opus file if possible. The first couple of MB should be enough.

comment:9 Changed 2 years ago by mbunkus

  • Cc moritz@bunkus.org added

Changed 2 years ago by agressiv

Changed 2 years ago by agressiv

Changed 2 years ago by agressiv

Changed 2 years ago by agressiv

Changed 2 years ago by agressiv

Changed 2 years ago by agressiv

Changed 2 years ago by agressiv

Changed 2 years ago by agressiv

comment:10 follow-up: Changed 2 years ago by agressiv

Ok, here is the explanation for the above files:

st6.mka/opus
Original Audio is encoded from TrueHD inside Source.mkv. Source.mkv is about 32gb.

ffmpeg.exe -i Source.mkv -acodec libopus -b:a 256k Audio.opus

Source.mkv contains extremely high-bitrate Video. I shrink that down with x264-x64.exe, and re-mux the file with mkvmerge.exe, which produces st6.mkv.

mkvmerge.exe -o st6.mkv -d 0 -A -S st6.264 --language 0:eng --sync 0:0 -a 0 -D -S Audio.opus --chapters st6.xml

I then use ffmpeg to extract 20 seconds of audio, with the following command:

ffmpeg.exe -i st6.mkv -acodec copy -ss 00:00:00.0 -c copy -t 00:00:20.0 st6.opus
ffmpeg.exe -i st6.mkv -acodec copy -ss 00:00:00.0 -c copy -t 00:00:20.0 st6.mka

The MKA file crackles at two points, whereas the Opus file does not.

st7.mka/opus
I use ffmpeg to extract opus from the original mkv encoding I did several months ago.

ffmpeg.exe -i st7.mkv -acodec copy -ss 00:00:00.0 -c copy -t 00:00:20.0 st7.opus
ffmpeg.exe -i st7.mkv -acodec copy -ss 00:00:00.0 -c copy -t 00:00:20.0 st7.mka

st7.mka is obviously very similar to the original attachment; st7.opus is extracted from the mkv rather than the mka.

The original Opus file which created st7.mkv is long gone.

I tried lots of experimentation to see if I could get inconsistent results, such as using multiple computers, stressing out the disk when doing the audio encoding and muxing - nothing would cause it to behave inconsistently.

st8.mka/opus (Not Uploaded)
I encoded these directly from the 32gb original. There were no artifacts. Note that mkvmerge is never in the equation either.

ffmpeg.exe -i Source.mkv -acodec libopus -b:a 256k -ss 00:00:00.0 -t 00:00:20.0 st8.mka
ffmpeg.exe -i Source.mkv -acodec libopus -b:a 256k -ss 00:00:00.0 -t 00:00:20.0 st8.opus

These were all with the old ffmpeg version I have, ffmpeg version N-67289-g7f24e1e from October.

st9.mka/opus
I encoded these just like st6.mka/opus, with mkvmerge, except with the latest ffmpeg binary, ffmpeg version N-69190-g6c559a0. This actually sounds worse, more artifacts, but not as bad as st7. Opus file plays ok.

st10.mka/opus
When I mux with ffmpeg instead of mkvmerge, I am not getting any problems:

ffmpeg.exe -i st10.264 -i Audio.opus -c copy -map 0:v:0 -map 1:a:0 -shortest st10.mkv
ffmpeg.exe -i st10.mkv -acodec copy -ss 00:00:00.0 -c copy -t 00:00:20.0 st10.opus
ffmpeg.exe -i st10.mkv -acodec copy -ss 00:00:00.0 -c copy -t 00:00:20.0 st10.mka

So, this still points in the direction of mkvmerge, but since I do not really know how to analyze these files the way you guys can, I won't jump to any other conclusions.

I've spent many hours on this now, and I'm sorry about all of the attachments, but I am very detail-oriented. If there is still something else I can do to help narrow this down, let me know.

Last edited 2 years ago by agressiv (previous) (diff)

comment:11 Changed 2 years ago by Timothy_Gu

  • Component changed from undetermined to avformat
  • Priority changed from important to wish
  • Type changed from defect to enhancement

comment:12 in reply to: ↑ 10 Changed 2 years ago by Timothy_Gu

Replying to agressiv:

Thanks for all the samples. I am sure they would be helpful.

So, this still points in the direction of mkvmerge, but since I do not really know how to analyze these files the way you guys can, I won't jump to any other conclusions.

Remember that there is always a possibility that there are two bugs in FFmpeg, one in the muxer and one in the demuxer, each covering up for the other. Such problems have been seen in the past where some files are only decodable by FFmpeg.

comment:13 Changed 2 years ago by mbunkus

This sounds like an issue in ffmpeg's Ogg or Opus output code. If I follow the steps you've described then I do see several more Ogg packets for which the expected granulepos doesn't match the Ogg page's actual granulepos.

What I've done:

  1. Merged a WAV file with mkvmerge, e.g. mkvmerge -o source.mka source.wav
  2. Encoded that to Ogg/Opus with ffmpeg, e.g. ffmpeg -i source.mka source.ogg
  3. Merged that Ogg/Opus file to an .mka and turned on debugging output during this conversion: mkvmerge --debug opus -o result.mka source.ogg

The debug output contains a couple of lines which report a difference between the actual granulepos and the expected one. Such lines look like this:

Debug> src/input/r_ogm.cpp:1134: Opus discard padding calculated 00:00:03.020000000 Ogg timestamp 00:00:03.019979166 diff 00:00:00.000020834 samples 1 (Ogg page's granulepos 144959)

(Note that the latest MKVToolNix release, v7.5.0, doesn't output the Ogg granulepos; that's a change I've just pushed to the MKVToolNix repo.)

Like I said: mkvmerge calculates the expected timestamp as the sum of the durations of all the Opus packets encountered so far by decoding their TOCs. This timestamp is calculated with the actual timestamp derived from the Ogg page's granulepos according to the official formula: timestamp_in_ns = 1_000_000_000 * granulepos / 48000 (note that pre-skip is not used in this calculation as the pre-skip is kept as a separate header value in Matroska).

Having said that this looks like a rounding error in the program (ffmpeg) creating the Ogg stream. That granulepos for the Ogg packet shown above is most likely supposed to be 144960 instead of 144959 as 144960 would result in a time stamp of exactly 3.02s – which would match the timestamp calculated by summing all the packet's durations by decoding the packet duration in the TOC.

comment:14 Changed 2 years ago by gjdfgh

Type changed from defect to enhancement

What???

comment:15 Changed 2 years ago by richardpl

  • Priority changed from wish to important
  • Type changed from enhancement to defect

comment:16 Changed 17 months ago by michael

  • Cc michael added

comment:17 Changed 8 weeks ago by anthontex

Good morning, I can confirm that this bug is still around.
I can reproduce this problem only with 7.1 tracks (now <=5.1 at least for me seems to work) and if and only if i'm encoding directly from an mkv container. Example:

ffmpeg -i "myfile.mkv" -af "channelmap=channel_layout=7.1" -c:a libopus -b:a 384K -mapping_family 1 out.opus 

Now that out.opus plays well with ffplay but if i mux it with indifferently ffmpeg or mkvtoolnix, audio is crackling and with artifacts.

Instead if i do this (let's suppose myfile has a DTS stream):

ffmpeg -i "myfile.mkv" -c:a copy out.dts
ffmpeg -i "out.dts" -af "channelmap=channel_layout=7.1" -c:a libopus -b:a 384K -mapping_family 1 out.opus 

That ogg/opus stream works fine now, also if i mux it in a mkv container.

Another thing that works is this:

Let's suppose i'm encoding as described in the first example ( that produce corrupted out.opus)
and put that track in a mkv container and so i'm getting artifacts:

ffmpeg -i "myfile.mkv" -af "channelmap=channel_layout=7.1" -c:a libopus -b:a 384K -mapping_family 1 out.opus
ffmpeg -i out.opus -c copy muxed.mkv

Now if do this:

ffmpeg -i muxed.mkv -c:a copy out2.opus
ffmpeg -i out2.opus -c copy muxed2.mkv

So if i demux it and remux, it seems that ffmpeg fix something and muxed2.mkv is now working well.

Note: See TracTickets for help on using tickets.