Opened 8 months ago

Last modified 8 months ago

#9641 new defect

Matroska duplicate tags are only output once instead of repeated

Reported by: the-black-eagle Owned by:
Priority: normal Component: avformat
Version: unspecified Keywords: matroska metadata
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description (last modified by the-black-eagle)

When using ffprobe from the commandline, or calling into ffmpeg using

while ((tag=av_dict_get(m_fctx->chapters[i]->metadata, "", tag, AV_DICT_IGNORE_SUFFIX)))

if there are duplicate tags, only the last tag and value is returned. The Matroska tag spec (https://www.matroska.org/technical/tagging.html) states that if a tag has multiple values, the tag should be repeated with one value per tag.

However, ffmpeg only returns the last tag and value. This prevents extending mka tag support in Kodi (which uses ffmpeg for tag reading) as there can be multiple composers/lyricists etc for individual tracks.

This bug is present in v4.4 (current version used by Kodi) and v5.0 using a static build of github master branch from https://johnvansickle.com/ffmpeg/ dated 20220108

ffprobe output (commandline ffprobe -v 9 -loglevel 99 -i ../music/5.1SurroundSoundTestFilesVariousFormatsAACAC3MP4DTSWAV/mka/Pink\ Floyd\ Live\ at\ Knebworth\ 1990.mka)

  ./ffprobe -v 9 -loglevel 99 -i ../music/5.1SurroundSoundTestFilesVariousFormatsAACAC3MP4DTSWAV/mka/Pink\ Floyd\ Live\ at\ Knebworth\ 1990.mka 
ffprobe version N-60236-gffb000fff8-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
  libavutil      57. 18.100 / 57. 18.100
  libavcodec     59. 20.100 / 59. 20.100
  libavformat    59. 17.100 / 59. 17.100
  libavdevice    59.  5.100 / 59.  5.100
  libavfilter     8. 25.100 /  8. 25.100
  libswscale      6.  5.100 /  6.  5.100
  libswresample   4.  4.100 /  4.  4.100
  libpostproc    56.  4.100 / 56.  4.100
[NULL @ 0x5a771c0] Opening '../music/5.1SurroundSoundTestFilesVariousFormatsAACAC3MP4DTSWAV/mka/Pink Floyd Live at Knebworth 1990.mka' for reading
[file @ 0x5a77a80] Setting default whitelist 'file,crypto,data'
Probing matroska,webm score:100 size:2048
[matroska,webm @ 0x5a771c0] Format matroska,webm probed with size=2048 and score=100
[matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013792645
[matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013792741
[matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013793267
[matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013793369
[matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013793535
[matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013794075
[matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013794240
[matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013794483
[matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013794666
[matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013794841
[matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013794962
st:0 removing common factor 32 from timebase
[matroska,webm @ 0x5a771c0] Before avformat_find_stream_info() pos: 5977 bytes read:35491 seeks:2 nb_streams:1
[matroska,webm @ 0x5a771c0] All info found
[matroska,webm @ 0x5a771c0] stream 0: start_time: 0 duration: NOPTS
[matroska,webm @ 0x5a771c0] format: start_time: 0 duration: 3426.7 (estimate from stream) bitrate=4701 kb/s
[matroska,webm @ 0x5a771c0] After avformat_find_stream_info() pos: 32664 bytes read:68259 seeks:2 frames:1
Input #0, matroska,webm, from '../music/5.1SurroundSoundTestFilesVariousFormatsAACAC3MP4DTSWAV/mka/Pink Floyd Live at Knebworth 1990.mka':
  Metadata:
    creation_time   : 2022-02-09T11:24:44.000000Z
    encoder         : libebml v1.3.10 + libmatroska v1.5.2
    TITLE           : Live at Knebworth 1990
    ALBUMARTISTSORT : Pink Floyd
    ARTIST          : Pink Floyd
    ARTISTSORT      : Pink Floyd
    MUSICBRAINZ_ALBUMARTIST: Pink Floyd
    MUSICBRAINZ_ALBUMARTISTID: 83d91898-7763-47d7-b03b-b92132375c47
    MUSICBRAINZ_ALBUMID: 4f6f7e13-1d87-430e-b2c7-86acf58fe15f
    MUSICBRAINZ_ARTISTID: 83d91898-7763-47d7-b03b-b92132375c47
    MUSICBRAINZ_RELEASEGROUPID: 9c4299b6-7e4a-4879-9fcf-5b15ca10a233
    PUBLISHER       : Parlophone
    DATE_RECORDED   : 2021
  Duration: 00:57:06.70, start: 0.000000, bitrate: 4701 kb/s
  Chapters:
    Chapter #0:0: start 0.000000, end 663.663000
      Metadata:
        MUSICBRAINZ_TRACKID: 84e3cf9a-f1f2-49ff-8d8a-8596d57a0a7a
        COMPOSER        : David Gilmour;Roger Waters;Richard Wright
        INSTRUMENTS     : David Gilmour,Guitar,Nick Mason,Drums,Guy Pratt,Bass Guitar,Clare Torry,Backing Vocals,Sam Brown,BackingVocals,Candy Dulfer,Saxaphone,Tim Renwick,Guitar,Richard Wright,Backing vocals,Richard Wright,keyboards
        LYRICIST        : Roger Waters
        TITLE           : Shine On You Crazy Diamond, Parts 1–5
        track           : 1/7
    Chapter #0:1: start 663.663000, end 967.633333
      Metadata:
        MUSICBRAINZ_TRACKID: 3cbc4c9c-bac2-47c3-a4fb-4515efe726c2
        COMPOSER        : Richard Wright
        TITLE           : The Great Gig in the Sky
        track           : 2/7
    Chapter #0:2: start 967.633333, end 1270.060458
      Metadata:
        MUSICBRAINZ_TRACKID: c2e10557-3208-4965-88c6-40c534cc0cfb
        TITLE           : Wish You Were Here
        track           : 3/7
    Chapter #0:3: start 1270.060458, end 1840.839000
      Metadata:
        MUSICBRAINZ_TRACKID: b5b91cd2-b0f1-4569-99b5-8ddce4e7c441
        TITLE           : Sorrow
        track           : 4/7
    Chapter #0:4: start 1840.839000, end 2453.159042
      Metadata:
        MUSICBRAINZ_TRACKID: 46725dce-1e11-4f29-a3c1-ce62168204c4
        TITLE           : Money
        track           : 5/7
    Chapter #0:5: start 2453.159042, end 2947.569625
      Metadata:
        MUSICBRAINZ_TRACKID: 366f18f0-c8fb-4a05-89ff-0ce83f1e289f
        TITLE           : Comfortably Numb
        track           : 6/7
    Chapter #0:6: start 2947.569625, end 3426.690000
      Metadata:
        MUSICBRAINZ_TRACKID: 28636bbd-b8ec-4cb0-9c3c-4444a4df7eb1
        TITLE           : Run Like Hell
        track           : 7/7
  Stream #0:0(eng), 1, 651/31250000: Audio: dts (DTS-HD MA), 48000 Hz, 5.1(side), s32p (24 bit)
    Metadata:
      title           : Surround 5.1
      BPS-eng         : 4699476
      DURATION-eng    : 00:57:06.699323040
      NUMBER_OF_BYTES-eng: 2012961516
      NUMBER_OF_FRAMES-eng: 321253
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_WRITING_APP-eng: mkvmerge v45.0.0 ('Heaven in Pennies') 64-bit
      _STATISTICS_WRITING_DATE_UTC-eng: 2022-02-09 11:24:44
[AVIOContext @ 0x5a7ff00] Statistics: 68259 bytes read, 2 seeks

Note that Chapter #0:1 has only one composer tag

Chapter #0:1: start 663.663000, end 967.633333

Metadata:

MUSICBRAINZ_TRACKID: 3cbc4c9c-bac2-47c3-a4fb-4515efe726c2
COMPOSER : Richard Wright
TITLE : The Great Gig in the Sky
track : 2/7

Output from mkvextract tags Pink\ Floyd\ Live\ at\ Knebworth\ 1990.mka > pftags.xml

  <?xml version="1.0"?>
<!-- <!DOCTYPE Tags SYSTEM "matroskatags.dtd"> -->
<Tags>
  <Tag>
    <Targets>
      <TargetTypeValue>50</TargetTypeValue>
    </Targets>
    <Simple>
      <Name>TITLE</Name>
      <String>Live at Knebworth 1990</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
      <TagLanguageIETF>und</TagLanguageIETF>
    </Simple>
    <Simple>
      <Name>ARTIST</Name>
      <String>Pink Floyd</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
  </Tag>
  <Tag>
    <Targets>
      <TargetTypeValue>30</TargetTypeValue>
    </Targets>
    <Simple>
      <Name>ALBUMARTISTSORT</Name>
      <String>Pink Floyd</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
      <TagLanguageIETF>und</TagLanguageIETF>
    </Simple>
    <Simple>
      <Name>ARTIST</Name>
      <String>Pink Floyd</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>ARTISTSORT</Name>
      <String>Pink Floyd</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>MUSICBRAINZ_ALBUMARTIST</Name>
      <String>Pink Floyd</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>MUSICBRAINZ_ALBUMARTISTID</Name>
      <String>83d91898-7763-47d7-b03b-b92132375c47</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>MUSICBRAINZ_ALBUMID</Name>
      <String>4f6f7e13-1d87-430e-b2c7-86acf58fe15f</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>MUSICBRAINZ_ARTISTID</Name>
      <String>83d91898-7763-47d7-b03b-b92132375c47</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>MUSICBRAINZ_RELEASEGROUPID</Name>
      <String>9c4299b6-7e4a-4879-9fcf-5b15ca10a233</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>PUBLISHER</Name>
      <String>Parlophone</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
  </Tag>
  <Tag>
    <Targets>
      <TargetTypeValue>50</TargetTypeValue>
    </Targets>
    <Simple>
      <Name>DATE_RECORDED</Name>
      <String>2021</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
      <TagLanguageIETF>und</TagLanguageIETF>
    </Simple>
  </Tag>
  <Tag>
    <Targets>
      <ChapterUID>1518495949379077925</ChapterUID>
      <TargetTypeValue>30</TargetTypeValue>
    </Targets>
    <Simple>
      <Name>MUSICBRAINZ_TRACKID</Name>
      <String>28636bbd-b8ec-4cb0-9c3c-4444a4df7eb1</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
      <TagLanguageIETF>und</TagLanguageIETF>
    </Simple>
    <Simple>
      <Name>TITLE</Name>
      <String>Run Like Hell</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>PART_NUMBER</Name>
      <String>7/7</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
  </Tag>
  <Tag>
    <Targets>
      <ChapterUID>3613098625400000886</ChapterUID>
      <TargetTypeValue>30</TargetTypeValue>
    </Targets>
    <Simple>
      <Name>COMPOSER</Name>
      <String>David Gilmour;Roger Waters;Richard Wright</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
      <TagLanguageIETF>und</TagLanguageIETF>
    </Simple>
    <Simple>
      <Name>INSTRUMENTS</Name>
      <String>David Gilmour,Guitar,Nick Mason,Drums,Guy Pratt,Bass Guitar,Clare Torry,Backing Vocals,Sam Brown,BackingVocals,Candy Dulfer,Saxaphone,Tim Renwick,Guitar,Richard Wright,Backing vocals,Richard Wright,keyboards</String>
    </Simple>
    <Simple>
      <Name>LYRICIST</Name>
      <String>Roger Waters</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>MUSICBRAINZ_TRACKID</Name>
      <String>84e3cf9a-f1f2-49ff-8d8a-8596d57a0a7a</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>TITLE</Name>
      <String>Shine On You Crazy Diamond, Parts 1–5</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>PART_NUMBER</Name>
      <String>1/7</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
  </Tag>
  <Tag>
    <Targets>
      <ChapterUID>11822505076461640796</ChapterUID>
      <TargetTypeValue>30</TargetTypeValue>
    </Targets>
    <Simple>
      <Name>MUSICBRAINZ_TRACKID</Name>
      <String>b5b91cd2-b0f1-4569-99b5-8ddce4e7c441</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
      <TagLanguageIETF>und</TagLanguageIETF>
    </Simple>
    <Simple>
      <Name>TITLE</Name>
      <String>Sorrow</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>PART_NUMBER</Name>
      <String>4/7</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
  </Tag>
  <Tag>
    <Targets>
      <ChapterUID>12948420648829714117</ChapterUID>
      <TargetTypeValue>30</TargetTypeValue>
    </Targets>
    <Simple>
      <Name>MUSICBRAINZ_TRACKID</Name>
      <String>46725dce-1e11-4f29-a3c1-ce62168204c4</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
      <TagLanguageIETF>und</TagLanguageIETF>
    </Simple>
    <Simple>
      <Name>TITLE</Name>
      <String>Money</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>PART_NUMBER</Name>
      <String>5/7</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
  </Tag>
  <Tag>
    <Targets>
      <ChapterUID>14931913620754867571</ChapterUID>
      <TargetTypeValue>30</TargetTypeValue>
    </Targets>
    <Simple>
      <Name>COMPOSER</Name>
      <String>Clare Torry</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>COMPOSER</Name>
      <String>Richard Wright</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>MUSICBRAINZ_TRACKID</Name>
      <String>3cbc4c9c-bac2-47c3-a4fb-4515efe726c2</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
      <TagLanguageIETF>und</TagLanguageIETF>
    </Simple>
    <Simple>
      <Name>TITLE</Name>
      <String>The Great Gig in the Sky</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>PART_NUMBER</Name>
      <String>2/7</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
  </Tag>
  <Tag>
    <Targets>
      <ChapterUID>16908591664885573413</ChapterUID>
      <TargetTypeValue>30</TargetTypeValue>
    </Targets>
    <Simple>
      <Name>MUSICBRAINZ_TRACKID</Name>
      <String>366f18f0-c8fb-4a05-89ff-0ce83f1e289f</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
      <TagLanguageIETF>und</TagLanguageIETF>
    </Simple>
    <Simple>
      <Name>TITLE</Name>
      <String>Comfortably Numb</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>PART_NUMBER</Name>
      <String>6/7</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
  </Tag>
  <Tag>
    <Targets>
      <ChapterUID>17396753124889256807</ChapterUID>
      <TargetTypeValue>30</TargetTypeValue>
    </Targets>
    <Simple>
      <Name>MUSICBRAINZ_TRACKID</Name>
      <String>c2e10557-3208-4965-88c6-40c534cc0cfb</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
      <TagLanguageIETF>und</TagLanguageIETF>
    </Simple>
    <Simple>
      <Name>TITLE</Name>
      <String>Wish You Were Here</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
    <Simple>
      <Name>PART_NUMBER</Name>
      <String>3/7</String>
      <TagLanguage>und</TagLanguage>
      <DefaultLanguage>1</DefaultLanguage>
    </Simple>
  </Tag>
  <Tag>
    <Targets>
      <TrackUID>3</TrackUID>
      <TargetTypeValue>50</TargetTypeValue>
    </Targets>
    <Simple>
      <Name>BPS</Name>
      <String>4699476</String>
      <TagLanguage>eng</TagLanguage>
      <TagLanguageIETF>en</TagLanguageIETF>
    </Simple>
    <Simple>
      <Name>DURATION</Name>
      <String>00:57:06.699323040</String>
      <TagLanguage>eng</TagLanguage>
    </Simple>
    <Simple>
      <Name>NUMBER_OF_BYTES</Name>
      <String>2012961516</String>
      <TagLanguage>eng</TagLanguage>
    </Simple>
    <Simple>
      <Name>NUMBER_OF_FRAMES</Name>
      <String>321253</String>
      <TagLanguage>eng</TagLanguage>
    </Simple>
    <Simple>
      <Name>_STATISTICS_TAGS</Name>
      <String>BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES</String>
      <TagLanguage>eng</TagLanguage>
    </Simple>
    <Simple>
      <Name>_STATISTICS_WRITING_APP</Name>
      <String>mkvmerge v45.0.0 ('Heaven in Pennies') 64-bit</String>
      <TagLanguage>eng</TagLanguage>
    </Simple>
    <Simple>
      <Name>_STATISTICS_WRITING_DATE_UTC</Name>
      <String>2022-02-09 11:24:44</String>
      <TagLanguage>eng</TagLanguage>
    </Simple>
  </Tag>
</Tags>

The above shows two composer tags with different values, whereas ffprobe/avformat only shows/returns the last tag and value.

<Tag>
  <Targets>
    <ChapterUID>14931913620754867571</ChapterUID>
    <TargetTypeValue>30</TargetTypeValue>
  </Targets>
  <Simple>
    <Name>COMPOSER</Name>
    <String>Clare Torry</String>
    <TagLanguage>und</TagLanguage>
    <DefaultLanguage>1</DefaultLanguage>
  </Simple>
  <Simple>
    <Name>COMPOSER</Name>
    <String>Richard Wright</String>
    <TagLanguage>und</TagLanguage>
    <DefaultLanguage>1</DefaultLanguage>
  </Simple>
  <Simple>
    <Name>MUSICBRAINZ_TRACKID</Name>
    <String>3cbc4c9c-bac2-47c3-a4fb-4515efe726c2</String>
    <TagLanguage>und</TagLanguage>
    <DefaultLanguage>1</DefaultLanguage>
    <TagLanguageIETF>und</TagLanguageIETF>
  </Simple>
  <Simple>
    <Name>TITLE</Name>
    <String>The Great Gig in the Sky</String>
    <TagLanguage>und</TagLanguage>
    <DefaultLanguage>1</DefaultLanguage>
  </Simple>
  <Simple>
    <Name>PART_NUMBER</Name>
    <String>2/7</String>
    <TagLanguage>und</TagLanguage>
    <DefaultLanguage>1</DefaultLanguage>
  </Simple>
</Tag>

If necessary I can supply the original file or I can tag a dummy mka/mkv file with the same tags for testing purposes.

Change History (1)

comment:1 by the-black-eagle, 8 months ago

Description: modified (diff)
Note: See TracTickets for help on using tickets.