GoPro metadata not properly handled

[importon]

Summary of the bug: I'm trying to swap out the video track of a GoPro mp4 file (0.mp4) with another (1.mp4) while keeping all the GPMF metadata in the 1st GoPro file. It results in error message "[mp4 @ 000002a357fdef40] Could not find tag for codec none in stream #2, codec not currently supported in container Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument Stream mapping: Stream #0:0 -> #0:0 (copy) Stream #0:1 -> #0:1 (copy) Stream #0:2 -> #0:2 (copy) Stream #0:3 -> #0:3 (copy) Stream #0:4 -> #0:4 (copy)"
How to reproduce:

ffmpeg -i 0.MP4 -i 1.mp4 -map_metadata 0 -map 0 -movflags use_metadata_tags -c copy -copy_unknown GHO10772mux.MP4

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

[Carl Eugen Hoyos]

In general, swapping is not possible because ffmpeg is a transcoding tool, not a file editor.
Please test current FFmpeg git head and provide the command line you tested together with the complete, uncut console output and a short input sample to make this a valid ticket.

[importon]

Thanks for the follow up.

I can't make my sample files small enough(2.5mb)to attach them so please grab them from here.
​https://drive.google.com/drive/folders/1Sow7A_Wz_K4Ug-2RwX3YAqCAlAEDia2J?usp=sharing

I upgraded to ffmpeg version 20191028-68f623d and ran the following with the results below.

ffmpeg -i 0.MP4 -i 1.mp4 -map_metadata 0 -map 0 -movflags use_metadata_tags -c copy -copy_unknown 01mux.mp4
ffmpeg version N-92498-g77bf85515e Copyright (c) 2000-2018 the FFmpeg developers

built with gcc 8.2.1 (GCC) 20181017
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
libavutil 56. 23.101 / 56. 23.101
libavcodec 58. 39.100 / 58. 39.100
libavformat 58. 22.100 / 58. 22.100
libavdevice 58. 6.100 / 58. 6.100
libavfilter 7. 46.100 / 7. 46.100
libswscale 5. 4.100 / 5. 4.100
libswresample 3. 4.100 / 3. 4.100
libpostproc 55. 4.100 / 55. 4.100

[mov,mp4,m4a,3gp,3g2,mj2 @ 000001c2556aa6c0] Using non-standard frame rate 59/1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '0.MP4':

Metadata:

major_brand : mp41
minor_version : 538120216
compatible_brands: mp41
creation_time : 2019-05-03T16:55:07.000000Z
firmware : HD6.01.01.60.00

Duration: 00:00:06.42, start: 0.000000, bitrate: 47516 kb/s

Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 47337 kb/s, 59.94 fps, 59.94 tbr, 60k tbn, 119.88 tbc (default)
Metadata:

creation_time : 2019-05-03T16:55:07.000000Z
handler_name : GoPro AVC
encoder : GoPro AVC encoder
timecode : 17:11:17:56

Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 188 kb/s (default)
Metadata:

creation_time : 2019-05-03T16:55:07.000000Z
handler_name : GoPro AAC
timecode : 17:11:17:56

Stream #0:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
Metadata:

creation_time : 2019-05-03T16:55:07.000000Z
handler_name : GoPro TCD
timecode : 17:11:17:56

Stream #0:3(eng): Data: bin_data (gpmd / 0x646D7067), 37 kb/s (default)
Metadata:

creation_time : 2019-05-03T16:55:07.000000Z
handler_name : GoPro MET

Stream #0:4(eng): Data: none (fdsc / 0x63736466), 14 kb/s (default)
Metadata:

creation_time : 2019-05-03T16:55:07.000000Z
handler_name : GoPro SOS

Input #1, mov,mp4,m4a,3gp,3g2,mj2, from '1.mp4':

Metadata:

major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.22.100

Duration: 00:00:06.42, start: 0.000000, bitrate: 101923 kb/s

Stream #1:0(eng): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv444p, 1920x1080 [SAR 1:1 DAR 16:9], 101967 kb/s, 59.94 fps, 59.94 tbr, 60k tbn, 119.88 tbc (default)
Metadata:

handler_name : GoPro AVC
timecode : 17:11:17:56

Stream #1:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 188 kb/s (default)
Metadata:

handler_name : GoPro AAC

Stream #1:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s
Metadata:

handler_name : GoPro AVC
timecode : 17:11:17:56

[mp4 @ 000001c2556aef00] You requested a copy of the original timecode track so timecode metadata are now ignored
[mp4 @ 000001c2556aef00] Could not find tag for codec none in stream #2, codec not currently supported in container
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Stream mapping:

Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Stream #0:2 -> #0:2 (copy)
Stream #0:3 -> #0:3 (copy)
Stream #0:4 -> #0:4 (copy)

Last message repeated 1 times

[Carl Eugen Hoyos]

I believe the following works as expected:

ffmpeg -i 0.MP4 -c copy -map 0:v -map 0:a -map 0:3 out.mp4

The fdsc tag is unsupported afaict.

[importon]

Thanks again for looking into this.

Unfortunately that does not copy all the GPMF metadata over. Is there anything in the works to handle GoPro GPMF data? More info here ​https://github.com/gopro/gpmf-parser.

[Jorge Toledo]

Hi. I just wanted to follow up on this. I got a similar error when trying to cut an .mp4 GoPro video with an app that relies on FFmpeg. According to the developer, "it seems that FFmpeg tries to process the data streams and fails to write that 'none' tag".

Here's the original issue so you can see the context: ​https://gitlab.gnome.org/YaLTeR/video-trimmer/-/issues/22

[importon]

Replying to eldelacajita:

Hi. I just wanted to follow up on this. I got a similar error when trying to cut an .mp4 GoPro video with an app that relies on FFmpeg. According to the developer, "it seems that FFmpeg tries to process the data streams and fails to write that 'none' tag".

Here's the original issue so you can see the context: ​https://gitlab.gnome.org/YaLTeR/video-trimmer/-/issues/22

If this is the case, would this be considered a bug that might be escalated to possibly being fixed? How does that process work?

[Roman Gaufman]

Any solution to this? - I tried:

ffmpeg -i 0.MP4 -c copy -map 0:v -map 0:a -map 0:3 out.mp4

But I get 4 steams in my output file instead of 5 in the original.

if I change to -map 0:4 it says:

$ ffmpeg -i GX010804\ NEW.MP4 -c copy -map 0:v -map 0:a -map 0:4 -y out.mp4
ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 12.0.5 (clang-1205.0.22.9)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/4.4_2 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-avresample --enable-videotoolbox
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'GX010804 NEW.MP4':
  Metadata:
    major_brand     : mp41
    minor_version   : 538120216
    compatible_brands: mp41
    creation_time   : 2021-04-24T16:42:06.000000Z
    firmware        : HD8.01.02.50.00
  Duration: 00:05:09.96, start: 0.000000, bitrate: 60283 kb/s
  Stream #0:0(eng): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, bt709), 3840x2160 [SAR 1:1 DAR 16:9], 60022 kb/s, 50 fps, 50 tbr, 90k tbn, 50 tbc (default)
    Metadata:
      creation_time   : 2021-04-24T16:42:06.000000Z
      handler_name    : GoPro H.265
      vendor_id       : [0][0][0][0]
      encoder         : GoPro H.265 encoder
      timecode        : 16:42:06:19
  Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 189 kb/s (default)
    Metadata:
      creation_time   : 2021-04-24T16:42:06.000000Z
      handler_name    : GoPro AAC
      vendor_id       : [0][0][0][0]
      timecode        : 16:42:06:19
  Stream #0:2(eng): Data: none (tmcd / 0x64636D74) (default)
    Metadata:
      creation_time   : 2021-04-24T16:42:06.000000Z
      handler_name    : GoPro TCD
      timecode        : 16:42:06:19
  Stream #0:3(eng): Data: bin_data (gpmd / 0x646D7067), 45 kb/s (default)
    Metadata:
      creation_time   : 2021-04-24T16:42:06.000000Z
      handler_name    : GoPro MET
  Stream #0:4(eng): Data: none (fdsc / 0x63736466), 12 kb/s (default)
    Metadata:
      creation_time   : 2021-04-24T16:42:06.000000Z
      handler_name    : GoPro SOS
[mp4 @ 0x12d81d000] Could not find tag for codec none in stream #2, codec not currently supported in container
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Error initializing output stream 0:2 --
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
  Stream #0:4 -> #0:2 (copy)
    Last message repeated 1 times
```

Any solution to this?

[Roman Gaufman]

Any solution/workaround for extracting/preserving GoPro data?

[Leseratte10]

I tested the latest version of FFmpeg and this issue still occurs, ffmpeg refuses to copy the GPMD metadata stream with the same error message "Could not find tag for codec none in stream #2, codec not currently supported in container" when working with MP4 files.

I can copy such a GoPro stream to other file formats (TS or MOV, for example) just fine, but that doesn't help me if I need to edit the original MP4 files coming from a GoPro camera.

I have also not found any other MP4 editing tool that can work with these files - but I can't imagine just having write support for that raw binary type could be that hard. FFmpeg already properly detects it and can extract it from a video, it just can't add it back to a video.

Is there any chance of FFmpeg getting support for writing GoPro metadata to an MP4 file?

Also, I'd agree with the statement in comment 10 that this should be a bug, not a feature request.

[Gabri.91]

Have you been able to find a workaround? I'm having the same issue with ffmpeg 6.1.1 and I'd like to keep gopro data in the MP4 file.

Thanks

[Kamil Filipek]

I have not found any workarounds for this problem.
copy_unknown, forcing a tag with tag:d:0, disabling write_tmcd (because this is the stream with which we are having issues with), none helps.
There is a mention of this issue (​https://superuser.com/a/1488253) not existing in an old version of ffmpeg tagged 4.0.2. But with my goal being re-encoding of the video stream into high quality HEVC, I see no reason in messing around with compilation of incompatible software.

I see a solution in adding a handler for this timecode stream into ffmpeg, or making it possible to copy data without known type (?codec?) tag, with copy_unknown.

I am looking into the code to find interesting diffs that caused the missing type tag fail the copy, but as someone not well acquainted with coding software like ffmpeg, I wouldn't hold my hopes up.

[Kamil Filipek]

I have seeked information on difference in code, between reported difference in handling the tmcd stream, by versions 4.0.2 & 4.2.1 with 4.2.1, being a version reporting the error Could not find tag for codec %s in stream #%d, codec not currently supported in container.

This error string is generated by code in libavformat/movenc.c. Diffing 4.2.1 with 4.0.2 reveals to me:

• In commit 981178f3b0b547a228804ff36641b2237eb75a16 streams with gpmd tag have been permitted into the MP4 container, as "GoPro Binary Data". (This is the stream which is detected by ffprobe and doesn't report any issues when copy is attempted)
• tmcd stream appears often associated with the .mov container and attempting to repackage the streams into .mov passes without issues.

Compared original .mp4 file

  Duration: 00:01:31.69, start: 0.000000, bitrate: 44716 kb/s
  Stream #0:0[0x1](eng): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, bt709), 3840x2160 [SAR 1:1 DAR 16:9], 44456 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
      Metadata:
        creation_time   : 2024-03-09T11:07:43.000000Z
        handler_name    : GoPro H.265
        vendor_id       : [0][0][0][0]
        encoder         : GoPro H.265 encoder
        timecode        : 11:07:03:02
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 189 kb/s (default)
      Metadata:
        creation_time   : 2024-03-09T11:07:43.000000Z
        handler_name    : GoPro AAC
        vendor_id       : [0][0][0][0]
        timecode        : 11:07:03:02
  Stream #0:2[0x3](eng): Data: none (tmcd / 0x64636D74) (default)
      Metadata:
        creation_time   : 2024-03-09T11:07:43.000000Z
        handler_name    : GoPro TCD
        timecode        : 11:07:03:02
  Stream #0:3[0x4](eng): Data: bin_data (gpmd / 0x646D7067), 52 kb/s (default)
      Metadata:
        creation_time   : 2024-03-09T11:07:43.000000Z
        handler_name    : GoPro MET

With .mov file in which only video stream should have been altered.

  Duration: 00:01:31.69, start: 0.000000, bitrate: 10166 kb/s
  Stream #0:0[0x1](eng): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, bt709, progressive), 3840x2160 [SAR 1:1 DAR 16:9], 9915 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
      Metadata:
        creation_time   : 2024-03-09T11:07:43.000000Z
        handler_name    : GoPro H.265
        vendor_id       : FFMP
        encoder         : Lavc61.3.100 hevc_nvenc
        timecode        : 11:07:03:02
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 189 kb/s (default)
      Metadata:
        creation_time   : 2024-03-09T11:07:43.000000Z
        handler_name    : GoPro AAC
        vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](eng): Data: none (tmcd / 0x64636D74) (default)
      Metadata:
        creation_time   : 2024-03-09T11:07:43.000000Z
        handler_name    : GoPro TCD
        timecode        : 11:07:03:02
  Stream #0:3[0x4](eng): Data: bin_data (gpmd / 0x646D7067), 52 kb/s (default)
      Metadata:
        creation_time   : 2024-03-09T11:07:43.000000Z
        handler_name    : GoPro MET

[Kamil Filipek]

On v4.0.2, the error is thrown when no track->par->codec_tag is found. For both .mov & .mp4 format.
The codec_tag is read in mov_find_codec_tag(s, track) function, with .mov going down to sub-function mov_get_codec_tag(s, track), where the same codec_tag parameter is read in case of unidentifiable codec.

In the branch where v4.2.1 has been released, a validate_codec_tag function has been introduced. It is not ran when MODE is set to MOV, but is executed for MODE of MP4 files.

The validate_codec_tag only does a simple check if the codec_tag or id exists in the white tables.

As I see the timecode track being a feature exclusive to Apple QuickTime & MOV container. I don't see any reason for it to receive preferential treatment.
But, I also don't see a reason for it to not be allowed a copy, if it already exists in the container and user explicitly accepts the risks with the copy_unknown flag.

So, I propose adoption of copy_unknown flag into the validate_codec_tag, where validation this validation is skipped and the stream is copied, just like in the case of MOV container. I'll attempt to make a valid commit for this change.

[MasterQuestionable]

͏    Such practice is alike the embedded subtitle streams... thus the serious failure.
͏    ​https://github.com/gopro/gpmf-parser#gpmf-introduction
͏    Essentially an issue regarding the whole container structure current.

[Kamil Filipek]

I am not sure if I understand you correctly @MasterQuestionable, but GPMF is handled correctly.
It is the TMCD stream which is not part of the MP4 container, but of QuickTime & MOV container. GoPro includes this stream anyway. For ?some?, hopefully, valid reason.

[MasterQuestionable]

͏    See also: ​https://github.com/MasterInQuestion/talk/discussions/3#discussioncomment-9376594

͏    The problem is that metadata self tends to be non-interoperable:
͏    At best mostly useless. (generally data ignorable: i.e. ideally removed as optimization)
͏    At times causes problems... (bug, side-channel of unintended information)

[Kamil Filipek]

I am not sure how we got from copying data streams between identical containers, to considering "ideal media container format", but I can not say it is not interesting.

From related information, I have figured out how to build statically linked FFMPEG binary on one of my servers (4G of RAM and limited storage being the biggest issues). It does take a while + my 500kb/s internet does not help in syphoning out a 140MB file.

Thus, hopefully, the tiny change I have introduced, is going to fulfil the task of permitting copy of the damn TMCD stream in GoPro files, so I can start re-encoding the ever growing GoPro footage library.

[Kamil Filipek]

I have tested the workaround and am experiencing no issues. Right now, sucessfuly re-encoding, N-th video.
Here is a release for anyone interested in playing with their GoPro footage (in my case re-encoding into more condensed H265)
​https://github.com/HighPriest/FFmpeg/releases/tag/Temp_GoPro_Workaround

The workaround IS NOT SAFE FOR PRODUCTION, as it is essentially skipping codec_validation function present in remuxer code.
The skip happens only for streams with unknown codec_tag, which shouldn't be mapped by ffmpeg anyway, if copy_unknown flag is not set. But I am only speculating based on my findings from one day.

[MasterQuestionable]

͏    How serious is the issue concerning contemporary GoPro?
͏    Recognized typical streams should be still properly handled..?
͏    Is keeping only the video, audio (via ͏"-map") adequately working for you?

͏    [ Quote Master ? @ CE 2024-03-18 13:18:03 UTC:
​https://bugzilla.mozilla.org/show_bug.cgi?id=1860417#c29
͏    "outside the scope"...
͏    不识 庐山 真面目: 只缘身在此山中.
+͏    Unable to tell Lú Mountain's true look: For self already in the mountain.
͏    .
͏    At many times the real problem is indeed "outside the scope". ]

[MasterQuestionable]

͏    Does comment:16 mean "-copy_unknown" doesn't just copy unknown and failed saying "unsupported" in current FFmpeg?

[Kamil Filipek]

Replying to MasterQuestionable:

͏    Does comment:16 mean "-copy_unknown" doesn't just copy unknown and failed saying "unsupported" in current FFmpeg?

Precisely. The FFmpeg_opts code handles the copy just fine (although, looking at the code, I don't see the copy_unknown option doing much). The streams we just want to copy, appear in the preview of output file structure.

But the re-muxing code present in movenc.c doesn't have any information about the copy_unknown option being set, so the codec validation code throws an error that the codec tag of the stream we want to copy is... unknown. This validation is not executed for .mov files, so you can do the copy and repackaging into MOV format just fine.