Unable to remux HEVC stream from MKV to MKV

[mysterious]

Summary of the bug:

With some MKV source files, copying the HEVC video stream to an MKV output may produce the error:

Could not write header for output file #0 (incorrect codec parameters ?): Invalid data found when processing input
Error initializing output stream 0:0 --

How to reproduce:

Run the following command with the attached sample.mkv:

ffmpeg -i sample.mkv -vcodec copy output.mkv

Tried on v4.4.1, but reproduced on Git Windows build 2022-02-07-git-04cc7a5548-essentials_build-www.gyan.dev

Full output:

ffmpeg version 2022-02-07-git-04cc7a5548-essentials_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 11.2.0 (Rev7, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
  libavutil      57. 21.100 / 57. 21.100
  libavcodec     59. 20.100 / 59. 20.100
  libavformat    59. 17.101 / 59. 17.101
  libavdevice    59.  5.100 / 59.  5.100
  libavfilter     8. 26.101 /  8. 26.101
  libswscale      6.  5.100 /  6.  5.100
  libswresample   4.  4.100 /  4.  4.100
  libpostproc    56.  4.100 / 56.  4.100
Input #0, matroska,webm, from 'b.mkv':
  Metadata:
    encoder         : libebml v1.3.7 + libmatroska v1.5.0
    creation_time   : 2022-02-10T10:46:36.000000Z
  Duration: 00:00:04.51, start: 0.000000, bitrate: 723 kb/s
  Stream #0:0: Video: hevc (Main), yuv420p(tv, bt709), 1920x1080, SAR 1:1 DAR 16:9, 23.98 fps, 23.98 tbr, 1k tbn (default)
    Metadata:
      title           : Presented By EMBER
      BPS-eng         : 707780
      DURATION-eng    : 00:00:04.505000000
      NUMBER_OF_FRAMES-eng: 108
      NUMBER_OF_BYTES-eng: 398569
      _STATISTICS_WRITING_APP-eng: mkvmerge v34.0.0 ('Sight and Seen') 64-bit
      _STATISTICS_WRITING_DATE_UTC-eng: 2022-02-10 10:46:36
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
File 'c.mkv' already exists. Overwrite? [y/N] y
Could not write header for output file #0 (incorrect codec parameters ?): Invalid data found when processing input
Error initializing output stream 0:0 --
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
    Last message repeated 1 times

Output from ffprobe -show_format -show_streams -print_format json sample.mkv

{
    "streams": [
        {
            "index": 0,
            "codec_name": "hevc",
            "codec_long_name": "H.265 / HEVC (High Efficiency Video Coding)",
            "profile": "Main",
            "codec_type": "video",
            "codec_tag_string": "[0][0][0][0]",
            "codec_tag": "0x0000",
            "width": 1920,
            "height": 1080,
            "coded_width": 1920,
            "coded_height": 1080,
            "closed_captions": 0,
            "film_grain": 0,
            "has_b_frames": 2,
            "sample_aspect_ratio": "1:1",
            "display_aspect_ratio": "16:9",
            "pix_fmt": "yuv420p",
            "level": 123,
            "color_range": "tv",
            "color_space": "bt709",
            "color_transfer": "bt709",
            "color_primaries": "bt709",
            "chroma_location": "left",
            "refs": 1,
            "r_frame_rate": "27021/1127",
            "avg_frame_rate": "27021/1127",
            "time_base": "1/1000",
            "start_pts": 0,
            "start_time": "0.000000",
            "extradata_size": 1975,
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0,
                "captions": 0,
                "descriptions": 0,
                "metadata": 0,
                "dependent": 0,
                "still_image": 0
            },
            "tags": {
                "title": "Presented By EMBER",
                "BPS-eng": "707780",
                "DURATION-eng": "00:00:04.505000000",
                "NUMBER_OF_FRAMES-eng": "108",
                "NUMBER_OF_BYTES-eng": "398569",
                "_STATISTICS_WRITING_APP-eng": "mkvmerge v34.0.0 ('Sight and Seen') 64-bit",
                "_STATISTICS_WRITING_DATE_UTC-eng": "2022-02-10 10:46:36",
                "_STATISTICS_TAGS-eng": "BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES"
            }
        }
    ],
    "format": {
        "filename": "sample.mkv",
        "nb_streams": 1,
        "nb_programs": 0,
        "format_name": "matroska,webm",
        "format_long_name": "Matroska / WebM",
        "start_time": "0.000000",
        "duration": "4.505000",
        "size": "407168",
        "bit_rate": "723050",
        "probe_score": 100,
        "tags": {
            "encoder": "libebml v1.3.7 + libmatroska v1.5.0",
            "creation_time": "2022-02-10T10:46:36.000000Z"
        }
    }
}

[mkver]

The extradata (CodecPrivate in Matroska parlance) of HEVC in Matroska is supposed to contain a HEVCDecoderConfigurationRecord and there is currently only one legal version of HEVCDecoderConfigurationRecord, namely version 1. Yet your file claims to be of version 0 (as the first byte of the CodecPrivate indicates). Simply remuxing this would lead to an invalid file (because version 0 is not valid).
If one changes the byte 0x00 at offset 4329 in your file by 0x01, remuxing succeeds (and the file seems fine).

[mysterious]

Thanks for the suggestion and quick response!

Is there some option/way to force ffmpeg to ignore such bad configurations and either generate an illegal file, or apply your suggested patch to make it legal?
I'm running this from a script with unknown inputs, so manually patching files isn't really feasible. ffmpeg already accepts it as input, and other tools (like mkvmerge) seem to handle it fine (I'm guessing they aren't as strict) so it would be nice if ffmpeg's strictness could be tuned to a similar level, despite the undesirable output it'd generate.

It seems like the '-err_detect' option only applies to input handling, not muxing, as setting it to 'ignore_err' still outputs an error. I couldn't find an option

[mkver]

There is no such option (and I don't think we should add an option to allow the creation of spec-incompliant files). And automatically fixing these files would mean that one would have to be sure that the version is the only thing that is wrong.

[mysterious]

Appreciate the response and suggestion!

And automatically fixing these files would mean that one would have to be sure that the version is the only thing that is wrong.

From what I can tell, the demuxer/decoder have no issues with the file (and doesn't even output a warning, from my test), which would suggest that nothing else (significant) is wrong. In other words, ffmpeg already is automatically fixing the stream, whether explicitly or implicitly (by ignoring the version).

It would seem logical to me that the muxer should apply this same fix (or otherwise operate in the same way as the demuxer) to be consistent with everything else, or have the demuxer reject the file entirely. The mismatch between demuxer and muxer here can be surprising to a user, and feels odd.

I'd imagine that if an input file is bad, the demuxer/decoder should be the gatekeeper here, instead of the muxer.

[mkver]

Replying to mysterious:

Appreciate the response and suggestion!

And automatically fixing these files would mean that one would have to be sure that the version is the only thing that is wrong.

From what I can tell, the demuxer/decoder have no issues with the file (and doesn't even output a warning, from my test), which would suggest that nothing else (significant) is wrong. In other words, ffmpeg already is automatically fixing the stream, whether explicitly or implicitly (by ignoring the version).

It would seem logical to me that the muxer should apply this same fix (or otherwise operate in the same way as the demuxer) to be consistent with everything else, or have the demuxer reject the file entirely. The mismatch between demuxer and muxer here can be surprising to a user, and feels odd.

I'd imagine that if an input file is bad, the demuxer/decoder should be the gatekeeper here, instead of the muxer.

This is completely wrong: The demuxer/decoder should handle everything they encounter as well as possible; but muxers should not output invalid files (this is called Postel's law). If we made the decoder more strict and error out on potentially playable, but out-of-spec content, the user will receive/see worse output; but if we allow to create out-of-spec files, these files will be more widespread.

[mysterious]

I get where you're coming from, though one could also say that these files exist because decoders accept them. Trying to accept everything leads to cases such as the infamous crazy HTML color parsing rules ​https://stackoverflow.com/questions/8318911 becoming standard, due to early browsers liberally accepting anything (forcing later browsers to adopt the same rules).

Of course, you could argue the issue either way, and I'm not here to debate which principle is better, just that one should keep in mind that ideals aren't universally the best approach.

I'm actually not suggesting that invalid files have to be generated, just that it makes sense for the fix ffmpeg already does be applied to its output.
In fact, ffmpeg already does this most of the time (where the stream isn't being copied), I'm just suggesting it should be applied in the one case where it doesn't.

As you suggest, not fixing a broken input means the user would receive worse output, and I don't see why users using the MKV muxer should be excluded here.
I don't mind if this "fixing functionality" is held behind a user-supplied flag. I know the 'hevc_mp4toannexb' filter works here, but it requires being more selective (have to check if input is HEVC), and may not apply to other scenarios with different codecs that could crop up.

Thanks again for the response!

[mkver]

The decoder does not "fix" anything; the decoder merely does not check the version. This makes sense for a decoder: After all, it has to be prepared for garbage in the non-version fields anyway.

[mysterious]

the decoder merely does not check the version

In other words, the decoder assumes the version is 1.
There may not be any specific code to set the version to 1, but it behaves exactly as if it had.

This is what I meant by an implicit fix - it fixes the problem by ignoring it. Whether there's specifically code to fix anything is irrelevant in the grand scheme, as it behaves as if a fix was put in place.

Even if you disagree with the above, I think a 'make it work as best as you can' style switch can be useful for a number of scenarios.

Anyway, it seems clear to me that there's no interest in addressing this issue, so I see no reason to continue this debate. Although I disagree, I understand and respect your position, and appreciate the time you've taken to advise and respond. :)

[mkver]

You still don't seem to understand the issue: Just ignoring the version still creates invalid output files with a muxer.

[mysterious]

As I doubt there'll be any change regarding ffmpeg on this front, for future people who find this and are having the same issue, I've written ​a patch to address it.

If the HEVC stream would be rejected, and the version is 0, the patch will spit out a warning and change it to a 1. I've limited this to version=0, as opposed to any version (I presume that some tools output this specifically) as there seems to be some magic number detection in the code, so limiting the scope to reduce potentially unwanted side effects.