Opened 3 years ago
Last modified 11 months ago
#9641 new defect
Matroska duplicate tags are only output once instead of repeated
Reported by: | the-black-eagle | Owned by: | |
---|---|---|---|
Priority: | normal | Component: | avformat |
Version: | unspecified | Keywords: | matroska metadata |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description (last modified by )
When using ffprobe from the commandline, or calling into ffmpeg using
while ((tag=av_dict_get(m_fctx->chapters[i]->metadata, "", tag, AV_DICT_IGNORE_SUFFIX)))
if there are duplicate tags, only the last tag and value is returned. The Matroska tag spec (https://www.matroska.org/technical/tagging.html) states that if a tag has multiple values, the tag should be repeated with one value per tag.
However, ffmpeg only returns the last tag and value. This prevents extending mka tag support in Kodi (which uses ffmpeg for tag reading) as there can be multiple composers/lyricists etc for individual tracks.
This bug is present in v4.4 (current version used by Kodi) and v5.0 using a static build of github master branch from https://johnvansickle.com/ffmpeg/ dated 20220108
ffprobe output (commandline ffprobe -v 9 -loglevel 99 -i ../music/5.1SurroundSoundTestFilesVariousFormatsAACAC3MP4DTSWAV/mka/Pink\ Floyd\ Live\ at\ Knebworth\ 1990.mka)
./ffprobe -v 9 -loglevel 99 -i ../music/5.1SurroundSoundTestFilesVariousFormatsAACAC3MP4DTSWAV/mka/Pink\ Floyd\ Live\ at\ Knebworth\ 1990.mka ffprobe version N-60236-gffb000fff8-static https://johnvansickle.com/ffmpeg/ Copyright (c) 2007-2022 the FFmpeg developers built with gcc 8 (Debian 8.3.0-6) configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg libavutil 57. 18.100 / 57. 18.100 libavcodec 59. 20.100 / 59. 20.100 libavformat 59. 17.100 / 59. 17.100 libavdevice 59. 5.100 / 59. 5.100 libavfilter 8. 25.100 / 8. 25.100 libswscale 6. 5.100 / 6. 5.100 libswresample 4. 4.100 / 4. 4.100 libpostproc 56. 4.100 / 56. 4.100 [NULL @ 0x5a771c0] Opening '../music/5.1SurroundSoundTestFilesVariousFormatsAACAC3MP4DTSWAV/mka/Pink Floyd Live at Knebworth 1990.mka' for reading [file @ 0x5a77a80] Setting default whitelist 'file,crypto,data' Probing matroska,webm score:100 size:2048 [matroska,webm @ 0x5a771c0] Format matroska,webm probed with size=2048 and score=100 [matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013792645 [matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013792741 [matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013793267 [matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013793369 [matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013793535 [matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013794075 [matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013794240 [matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013794483 [matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013794666 [matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013794841 [matroska,webm @ 0x5a771c0] Unknown entry 0x447B at pos. 2013794962 st:0 removing common factor 32 from timebase [matroska,webm @ 0x5a771c0] Before avformat_find_stream_info() pos: 5977 bytes read:35491 seeks:2 nb_streams:1 [matroska,webm @ 0x5a771c0] All info found [matroska,webm @ 0x5a771c0] stream 0: start_time: 0 duration: NOPTS [matroska,webm @ 0x5a771c0] format: start_time: 0 duration: 3426.7 (estimate from stream) bitrate=4701 kb/s [matroska,webm @ 0x5a771c0] After avformat_find_stream_info() pos: 32664 bytes read:68259 seeks:2 frames:1 Input #0, matroska,webm, from '../music/5.1SurroundSoundTestFilesVariousFormatsAACAC3MP4DTSWAV/mka/Pink Floyd Live at Knebworth 1990.mka': Metadata: creation_time : 2022-02-09T11:24:44.000000Z encoder : libebml v1.3.10 + libmatroska v1.5.2 TITLE : Live at Knebworth 1990 ALBUMARTISTSORT : Pink Floyd ARTIST : Pink Floyd ARTISTSORT : Pink Floyd MUSICBRAINZ_ALBUMARTIST: Pink Floyd MUSICBRAINZ_ALBUMARTISTID: 83d91898-7763-47d7-b03b-b92132375c47 MUSICBRAINZ_ALBUMID: 4f6f7e13-1d87-430e-b2c7-86acf58fe15f MUSICBRAINZ_ARTISTID: 83d91898-7763-47d7-b03b-b92132375c47 MUSICBRAINZ_RELEASEGROUPID: 9c4299b6-7e4a-4879-9fcf-5b15ca10a233 PUBLISHER : Parlophone DATE_RECORDED : 2021 Duration: 00:57:06.70, start: 0.000000, bitrate: 4701 kb/s Chapters: Chapter #0:0: start 0.000000, end 663.663000 Metadata: MUSICBRAINZ_TRACKID: 84e3cf9a-f1f2-49ff-8d8a-8596d57a0a7a COMPOSER : David Gilmour;Roger Waters;Richard Wright INSTRUMENTS : David Gilmour,Guitar,Nick Mason,Drums,Guy Pratt,Bass Guitar,Clare Torry,Backing Vocals,Sam Brown,BackingVocals,Candy Dulfer,Saxaphone,Tim Renwick,Guitar,Richard Wright,Backing vocals,Richard Wright,keyboards LYRICIST : Roger Waters TITLE : Shine On You Crazy Diamond, Parts 1–5 track : 1/7 Chapter #0:1: start 663.663000, end 967.633333 Metadata: MUSICBRAINZ_TRACKID: 3cbc4c9c-bac2-47c3-a4fb-4515efe726c2 COMPOSER : Richard Wright TITLE : The Great Gig in the Sky track : 2/7 Chapter #0:2: start 967.633333, end 1270.060458 Metadata: MUSICBRAINZ_TRACKID: c2e10557-3208-4965-88c6-40c534cc0cfb TITLE : Wish You Were Here track : 3/7 Chapter #0:3: start 1270.060458, end 1840.839000 Metadata: MUSICBRAINZ_TRACKID: b5b91cd2-b0f1-4569-99b5-8ddce4e7c441 TITLE : Sorrow track : 4/7 Chapter #0:4: start 1840.839000, end 2453.159042 Metadata: MUSICBRAINZ_TRACKID: 46725dce-1e11-4f29-a3c1-ce62168204c4 TITLE : Money track : 5/7 Chapter #0:5: start 2453.159042, end 2947.569625 Metadata: MUSICBRAINZ_TRACKID: 366f18f0-c8fb-4a05-89ff-0ce83f1e289f TITLE : Comfortably Numb track : 6/7 Chapter #0:6: start 2947.569625, end 3426.690000 Metadata: MUSICBRAINZ_TRACKID: 28636bbd-b8ec-4cb0-9c3c-4444a4df7eb1 TITLE : Run Like Hell track : 7/7 Stream #0:0(eng), 1, 651/31250000: Audio: dts (DTS-HD MA), 48000 Hz, 5.1(side), s32p (24 bit) Metadata: title : Surround 5.1 BPS-eng : 4699476 DURATION-eng : 00:57:06.699323040 NUMBER_OF_BYTES-eng: 2012961516 NUMBER_OF_FRAMES-eng: 321253 _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES _STATISTICS_WRITING_APP-eng: mkvmerge v45.0.0 ('Heaven in Pennies') 64-bit _STATISTICS_WRITING_DATE_UTC-eng: 2022-02-09 11:24:44 [AVIOContext @ 0x5a7ff00] Statistics: 68259 bytes read, 2 seeks
Note that Chapter #0:1 has only one composer tag
Chapter #0:1: start 663.663000, end 967.633333
Metadata:
MUSICBRAINZ_TRACKID: 3cbc4c9c-bac2-47c3-a4fb-4515efe726c2
COMPOSER : Richard Wright
TITLE : The Great Gig in the Sky
track : 2/7
Output from mkvextract tags Pink\ Floyd\ Live\ at\ Knebworth\ 1990.mka > pftags.xml
<?xml version="1.0"?> <!-- <!DOCTYPE Tags SYSTEM "matroskatags.dtd"> --> <Tags> <Tag> <Targets> <TargetTypeValue>50</TargetTypeValue> </Targets> <Simple> <Name>TITLE</Name> <String>Live at Knebworth 1990</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> <TagLanguageIETF>und</TagLanguageIETF> </Simple> <Simple> <Name>ARTIST</Name> <String>Pink Floyd</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> </Tag> <Tag> <Targets> <TargetTypeValue>30</TargetTypeValue> </Targets> <Simple> <Name>ALBUMARTISTSORT</Name> <String>Pink Floyd</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> <TagLanguageIETF>und</TagLanguageIETF> </Simple> <Simple> <Name>ARTIST</Name> <String>Pink Floyd</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>ARTISTSORT</Name> <String>Pink Floyd</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>MUSICBRAINZ_ALBUMARTIST</Name> <String>Pink Floyd</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>MUSICBRAINZ_ALBUMARTISTID</Name> <String>83d91898-7763-47d7-b03b-b92132375c47</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>MUSICBRAINZ_ALBUMID</Name> <String>4f6f7e13-1d87-430e-b2c7-86acf58fe15f</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>MUSICBRAINZ_ARTISTID</Name> <String>83d91898-7763-47d7-b03b-b92132375c47</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>MUSICBRAINZ_RELEASEGROUPID</Name> <String>9c4299b6-7e4a-4879-9fcf-5b15ca10a233</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>PUBLISHER</Name> <String>Parlophone</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> </Tag> <Tag> <Targets> <TargetTypeValue>50</TargetTypeValue> </Targets> <Simple> <Name>DATE_RECORDED</Name> <String>2021</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> <TagLanguageIETF>und</TagLanguageIETF> </Simple> </Tag> <Tag> <Targets> <ChapterUID>1518495949379077925</ChapterUID> <TargetTypeValue>30</TargetTypeValue> </Targets> <Simple> <Name>MUSICBRAINZ_TRACKID</Name> <String>28636bbd-b8ec-4cb0-9c3c-4444a4df7eb1</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> <TagLanguageIETF>und</TagLanguageIETF> </Simple> <Simple> <Name>TITLE</Name> <String>Run Like Hell</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>PART_NUMBER</Name> <String>7/7</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> </Tag> <Tag> <Targets> <ChapterUID>3613098625400000886</ChapterUID> <TargetTypeValue>30</TargetTypeValue> </Targets> <Simple> <Name>COMPOSER</Name> <String>David Gilmour;Roger Waters;Richard Wright</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> <TagLanguageIETF>und</TagLanguageIETF> </Simple> <Simple> <Name>INSTRUMENTS</Name> <String>David Gilmour,Guitar,Nick Mason,Drums,Guy Pratt,Bass Guitar,Clare Torry,Backing Vocals,Sam Brown,BackingVocals,Candy Dulfer,Saxaphone,Tim Renwick,Guitar,Richard Wright,Backing vocals,Richard Wright,keyboards</String> </Simple> <Simple> <Name>LYRICIST</Name> <String>Roger Waters</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>MUSICBRAINZ_TRACKID</Name> <String>84e3cf9a-f1f2-49ff-8d8a-8596d57a0a7a</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>TITLE</Name> <String>Shine On You Crazy Diamond, Parts 1–5</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>PART_NUMBER</Name> <String>1/7</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> </Tag> <Tag> <Targets> <ChapterUID>11822505076461640796</ChapterUID> <TargetTypeValue>30</TargetTypeValue> </Targets> <Simple> <Name>MUSICBRAINZ_TRACKID</Name> <String>b5b91cd2-b0f1-4569-99b5-8ddce4e7c441</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> <TagLanguageIETF>und</TagLanguageIETF> </Simple> <Simple> <Name>TITLE</Name> <String>Sorrow</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>PART_NUMBER</Name> <String>4/7</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> </Tag> <Tag> <Targets> <ChapterUID>12948420648829714117</ChapterUID> <TargetTypeValue>30</TargetTypeValue> </Targets> <Simple> <Name>MUSICBRAINZ_TRACKID</Name> <String>46725dce-1e11-4f29-a3c1-ce62168204c4</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> <TagLanguageIETF>und</TagLanguageIETF> </Simple> <Simple> <Name>TITLE</Name> <String>Money</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>PART_NUMBER</Name> <String>5/7</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> </Tag> <Tag> <Targets> <ChapterUID>14931913620754867571</ChapterUID> <TargetTypeValue>30</TargetTypeValue> </Targets> <Simple> <Name>COMPOSER</Name> <String>Clare Torry</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>COMPOSER</Name> <String>Richard Wright</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>MUSICBRAINZ_TRACKID</Name> <String>3cbc4c9c-bac2-47c3-a4fb-4515efe726c2</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> <TagLanguageIETF>und</TagLanguageIETF> </Simple> <Simple> <Name>TITLE</Name> <String>The Great Gig in the Sky</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>PART_NUMBER</Name> <String>2/7</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> </Tag> <Tag> <Targets> <ChapterUID>16908591664885573413</ChapterUID> <TargetTypeValue>30</TargetTypeValue> </Targets> <Simple> <Name>MUSICBRAINZ_TRACKID</Name> <String>366f18f0-c8fb-4a05-89ff-0ce83f1e289f</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> <TagLanguageIETF>und</TagLanguageIETF> </Simple> <Simple> <Name>TITLE</Name> <String>Comfortably Numb</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>PART_NUMBER</Name> <String>6/7</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> </Tag> <Tag> <Targets> <ChapterUID>17396753124889256807</ChapterUID> <TargetTypeValue>30</TargetTypeValue> </Targets> <Simple> <Name>MUSICBRAINZ_TRACKID</Name> <String>c2e10557-3208-4965-88c6-40c534cc0cfb</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> <TagLanguageIETF>und</TagLanguageIETF> </Simple> <Simple> <Name>TITLE</Name> <String>Wish You Were Here</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>PART_NUMBER</Name> <String>3/7</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> </Tag> <Tag> <Targets> <TrackUID>3</TrackUID> <TargetTypeValue>50</TargetTypeValue> </Targets> <Simple> <Name>BPS</Name> <String>4699476</String> <TagLanguage>eng</TagLanguage> <TagLanguageIETF>en</TagLanguageIETF> </Simple> <Simple> <Name>DURATION</Name> <String>00:57:06.699323040</String> <TagLanguage>eng</TagLanguage> </Simple> <Simple> <Name>NUMBER_OF_BYTES</Name> <String>2012961516</String> <TagLanguage>eng</TagLanguage> </Simple> <Simple> <Name>NUMBER_OF_FRAMES</Name> <String>321253</String> <TagLanguage>eng</TagLanguage> </Simple> <Simple> <Name>_STATISTICS_TAGS</Name> <String>BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES</String> <TagLanguage>eng</TagLanguage> </Simple> <Simple> <Name>_STATISTICS_WRITING_APP</Name> <String>mkvmerge v45.0.0 ('Heaven in Pennies') 64-bit</String> <TagLanguage>eng</TagLanguage> </Simple> <Simple> <Name>_STATISTICS_WRITING_DATE_UTC</Name> <String>2022-02-09 11:24:44</String> <TagLanguage>eng</TagLanguage> </Simple> </Tag> </Tags>
The above shows two composer tags with different values, whereas ffprobe/avformat only shows/returns the last tag and value.
<Tag> <Targets> <ChapterUID>14931913620754867571</ChapterUID> <TargetTypeValue>30</TargetTypeValue> </Targets> <Simple> <Name>COMPOSER</Name> <String>Clare Torry</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>COMPOSER</Name> <String>Richard Wright</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>MUSICBRAINZ_TRACKID</Name> <String>3cbc4c9c-bac2-47c3-a4fb-4515efe726c2</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> <TagLanguageIETF>und</TagLanguageIETF> </Simple> <Simple> <Name>TITLE</Name> <String>The Great Gig in the Sky</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> <Simple> <Name>PART_NUMBER</Name> <String>2/7</String> <TagLanguage>und</TagLanguage> <DefaultLanguage>1</DefaultLanguage> </Simple> </Tag>
If necessary I can supply the original file or I can tag a dummy mka/mkv file with the same tags for testing purposes.
Change History (3)
comment:1 by , 3 years ago
Description: | modified (diff) |
---|
comment:2 by , 11 months ago
comment:3 by , 11 months ago
Observations:
- In
read_tag
(libavformat/ffmetadec.c
), the call toav_dict_set
should probably just use theflag, period.
- The invocation of
av_copy_dict
incopy_metadata
(fftools/ffmpeg_mux_init.c
) seems suspect. IIUC, the existing metadata of the output entity (stream, chapter, etc.) is preserved in favor of the input entity. This doesn't seem like the desired behavior? In any case, this needs to change somehow to support duplicate tags from the input.
- The above changes are an improvement, but duplicate tags are still getting lost somewhere. It's not obvious to me at this point where.
Stream mapping: Stream #0:0 -> #0:0 (copy) Output #0, opus, to 'redacted.opus': Metadata: TITLE : redacted TRACKNUMBER : 09 ALBUM : redacted ARTIST : redacted TRACKTOTAL : 12 METADATA_BLOCK_PICTURE: long base64 string METADATA_BLOCK_PICTURE: long base64 string encoder : Lavf61.0.100 Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default) Metadata: TITLE : redacted TRACKNUMBER : 09 ALBUM : redacted ARTIST : redacted TRACKTOTAL : 12 METADATA_BLOCK_PICTURE: long base64 string encoder : Lavf61.0.100
Is anyone working on this? It seems like it shouldn't be an overly difficult fix...
I have been searching (in vain, it seems) for a way to edit opus tags; ffmpeg almost works except for this bug, which seems to still be present in at least 6.0.1.
In my case, I specifically want to add multiple
METADATA_BLOCK_PICTURE
.