Ffmpeg not removing unwanted metadata

[steipal]

Summary of the bug:
FFmpeg don't seem to properly remove metadata even when specified.
The attached file has metadata or side data like:

Mastering Display Metadata, has_primaries:1 has_luminance:1 r(0.7080,0.2920) g(0.1700,0.7970) b(0.1310 0.0460) wp(0.3127, 0.3290) min_luminance=0.000000, max_luminance=9999.980500
Content Light Level Metadata, MaxCLL=10000, MaxFALL=5656

This needs to be removed from the output.

How to reproduce:

% ffmpeg -i sidedata.mp4 -vframes 1 -c:v libx264 -map_metadata -1 -vf sidedata=delete,metadata=delete -f null -

Full uncut console output:

ffmpeg version N-103405-g1930a85e83 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10.3.0 (Rev2, Built by MSYS2 project)
  configuration:  --disable-static --enable-shared --pkg-config=pkgconf --cc='ccache gcc' --cxx='ccache g++' --disable-autodetect --enable-amf --enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-nvenc --enable-zlib --enable-sdl2 --enable-ffnvcodec --enable-nvdec --enable-cuda-llvm --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libx264 --enable-libx265 --enable-libdav1d --enable-libaom --disable-debug --enable-fontconfig --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --enable-libxvid --enable-libopenmpt --enable-version3 --enable-libsrt --enable-libgsm --enable-libvmaf --enable-libsvtav1 --enable-mbedtls --extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++ --extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv --disable-w32threads --shlibdir=/local64/bin-video
  libavutil      57.  4.101 / 57.  4.101
  libavcodec     59.  6.100 / 59.  6.100
  libavformat    59.  4.102 / 59.  4.102
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8.  4.100 /  8.  4.100
  libswscale      6.  0.100 /  6.  0.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'sidedata.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf59.4.102
  Duration: 00:00:00.04, start: 0.000000, bitrate: 2217 kb/s
  Stream #0:0[0x1](eng): Video: vp9 (Profile 2) (vp09 / 0x39307076), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x1600, 2049 kb/s, SAR 1:1 DAR 12:5, 24.39 fps, 24.39 tbr, 16k tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
    Side data:
      Mastering Display Metadata, has_primaries:1 has_luminance:1 r(0.7080,0.2920) g(0.1700,0.7970) b(0.1310 0.0460) wp(0.3127, 0.3290) min_luminance=0.000000, max_luminance=9999.980500
      Content Light Level Metadata, MaxCLL=10000, MaxFALL=5656
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 000002b2666245c0] using SAR=1/1
[libx264 @ 000002b2666245c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
[libx264 @ 000002b2666245c0] profile High 10, level 5.1, 4:2:0, 10-bit
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf59.4.102
  Stream #0:0(eng): Video: h264, yuv420p10le(tv, bt2020nc/bt2020/smpte2084, progressive), 3840x1600 [SAR 1:1 DAR 12:5], q=2-31, 24.39 fps, 24.39 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc59.6.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
      Mastering Display Metadata, has_primaries:1 has_luminance:1 r(0.7080,0.2920) g(0.1700,0.7970) b(0.1310 0.0460) wp(0.3127, 0.3290) min_luminance=0.000000, max_luminance=9999.980500
      Content Light Level Metadata, MaxCLL=10000, MaxFALL=5656
frame=    1 fps=0.0 q=40.0 Lsize=N/A time=00:00:00.04 bitrate=N/A speed=0.195x
video:5kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 000002b2666245c0] frame I:1     Avg QP:21.80  size:  4755
[libx264 @ 000002b2666245c0] mb I  I16..4: 91.1%  8.3%  0.6%
[libx264 @ 000002b2666245c0] 8x8 transform intra:8.3%
[libx264 @ 000002b2666245c0] coded y,uvDC,uvAC intra: 0.4% 0.0% 0.0%
[libx264 @ 000002b2666245c0] i16 v,h,dc,p: 99%  0%  1%  0%
[libx264 @ 000002b2666245c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 86%  4% 10%  0%  0%  0%  0%  0%  0%
[libx264 @ 000002b2666245c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 41% 19% 19%  3%  3%  4%  3%  3%  3%
[libx264 @ 000002b2666245c0] i8c dc,h,v,p: 100%  0%  0%  0%
[libx264 @ 000002b2666245c0] kb/s:927.80

I would assume that "-map_metadata -1 -vf sidedata=delete,metadata=delete" would be enough to remove the unwanted metadata. If my assumptions are wrong I would appreciate instructions on how to remove the metadata.

[Francesco Bucciantini]

Yep, I confirm the issue as I can reproduce it.
I guess we're gonna have to wait for the ffmpeg devs to say something.

[Elon Musk]

metadata is not needed here. sidedata filter needs type option to be set.

Ticket is invalid.

[Elon Musk]

On second look, using those 2 filters is completely not supposed to remove those things.
So those 2 filters behave as expected, the thing is probably not implemented at all for the feature you really need.

[steipal]

First, if docs for the sidedata filter is correct:

"If type is not set, delete all side data in the frame."

The thing is that this side data will break some players utilizing it. And they are clearly wrong if one for example convert this source to 8 bit bt601.
Usually I'm forced to use sidedata=delete and -map_metadata -1 in order to get rid of unwanted side data and metadata, and this has fixed problems like this before. The side data in question is indeed presented as side data by ffmpeg and so one is lead to believe that they can also be removed with for example sidedata=delete. But they don't, and if there is no means to remove it then we have a problem.

PS: I'm fully aware that one can remove this by utilizing AviSynth or piping to another ffmpeg but this is clearly not a good solution.

[Francesco Bucciantini]

Hi there,
this is Frank from Sky.
Those properties being passed through is very concerning.
I tried to remux the MP4 created with FFMpeg with MP4Box and they were still there.
This is important 'cause we send out lots of MP4 to third party services, some of which are literally just TVs running our ads in shops where people generally go to buy a TV and given the overwhelmingly increasing number of TV sets supporting HDR, I'm not really sure what would happen if we give them an BT709 file with the metadata of HDR PQ like we're doing in this case.
Worse case scenario: they will interpret it LITERALLY so PQ transfer with the BT709 matrix and really bad things will happen.
This is not ok.
There should be a good standardized way in FFMpeg to get rid of those kind of metadata without going through Avisynth in the first place.

[pdr0]

A workaround is to pipe ffmpeg to ffmpeg . Sidedata is not transmitted through yuv4mpegpipe or rawvideo pipe

[steipal]

Replying to pdr0:

A workaround is to pipe ffmpeg to ffmpeg . Sidedata is not transmitted through yuv4mpegpipe or rawvideo pipe

As I wrote in comment no. 4, that's not a good solution.

[hydra3333]

Thank you for the report !
Since I regularly transcode home based clips; I also would like to see the facility to reliably remove metadata/sidedata and imagine that many if not all people using ffmpeg may "expect" such a thing from a swiss-knife type tool as great as ffmpeg.
Thank you.

[Elon Musk]

As already said multitude of times, this is neither frame metadata nor frame side data but this is packet side data, and thus no possible to remove with filters but only with bitstream filters.

Such bitstream filter, even trivial, is not implemented yet. And I'm not interest to implement if for free.

[steipal]

Fair enough. I would fix this myself for free if I had the knowhow to do so. The sad thing is that ffmpeg is insisting on keeping incompatible unneeded data and we have no generic way of removing it. In short: we need the ability to feed pure video/audio data to the filter and encoding process. Our only consistent option/workaround currently seem to be process piping or by using frame servers like AviSynth.

[steipal]

It looks like this following section in fftools\ffmpeg.c (line 3640) is responsible for the behavior that leads to incompatible side data in the output file:

        /*
         * Add global input side data. For now this is naive, and copies it
         * from the input stream's global side data. All side data should
         * really be funneled over AVFrame and libavfilter, then added back to
         * packet side data, and then potentially using the first packet for
         * global side data.
         */
        if (ist) {
            int i;
            for (i = 0; i < ist->st->nb_side_data; i++) {
                AVPacketSideData *sd = &ist->st->side_data[i];
                if (sd->type != AV_PKT_DATA_CPB_PROPERTIES) {
                    uint8_t *dst = av_stream_new_side_data(ost->st, sd->type, sd->size);
                    if (!dst)
                        return AVERROR(ENOMEM);
                    memcpy(dst, sd->data, sd->size);
                    if (ist->autorotate && sd->type == AV_PKT_DATA_DISPLAYMATRIX)
                        av_display_rotation_set((uint32_t *)dst, 0);
                }
            }
        }

I commented it out, compiled and ffmpeg no longer copied the side data. Also, when reading the comment it looks like this implementation is rather unfinished ad-hoc code that should not treat global packet side data this way. One thought was to link it to the "-map_metadata -1" option and when this is specified global side data will NOT be copied. However, I guess this implementation is rather ad-hoc too but from a user point of view it would make sense.

[riba101]

Is there any progress on this ticket? I'm running into the same issue.

I was playing back a x265 file that was tonemapped to rec709 and the player played is as rec2020 st2084 it looked like garbage.

[scjet63]

I'd also like to know if there's any update on this issue?

"-map_metadata -1" used to work on older ffmpeg versions, but lately, (< 2 years now) "-map_metadata -1" does not globally remove metadata anymore.
I've been compiling ffmpeg in linux for a number of years now, generally following
​https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu
I tried latest compile/re-compile of ffmpeg and "-map_metadata -1" still does not remove metadata.?
I can confirm this issue.

UPDATE:

I just tried the (static) ffmpeg build from ​https://johnvansickle.com/ffmpeg/ and also tried the ffmpeg build that comes with my Linux Mint 21.2 distro and they both were able to remove the metadata using "-map_metadata -1"

BEFORE: ffmpeg -i input.avi

---

...
Input #0, avi, from 'input.avi':
  Metadata:
    software        : Lavf59.37.100
  Duration: 00:49:27.20, start: 0.000000, bitrate: 966 kb/s
  Stream #0:0: Video: mpeg4 (DX50 / 0x30355844), yuv420p, 352x260 [SAR 1:1 DAR 88:65], 793 kb/s, 25 fps, 25 tbr, 25 tbn
  Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 44100 Hz, stereo, fltp, 160 kb/s
    Metadata:
      title           : Audio Stream

---

AFTER: ffmpeg -i input.avi -map_metadata -1 -map 0:0 -map 0:1 -c:v copy -c:a copy output.avi

---

...
Input #0, avi, from 'output.avi':
  Metadata:
    software        : Lavf60.18.100
  Duration: 00:49:27.20, start: 0.000000, bitrate: 966 kb/s
  Stream #0:0: Video: mpeg4 (DX50 / 0x30355844), yuv420p, 352x260 [SAR 1:1 DAR 88:65], 793 kb/s, 25 fps, 25 tbr, 25 tbn
  Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 44100 Hz, stereo, fltp, 160 kb/s

---

So my issue now is, how can I compile ffmpeg as per ​https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu guide, and still retain the ability to remove metadata using "-map_metadata -1"

Or, should I open a new ticket for this specific compiling problem of mine.?

Thanks ahead for any advice.