Opened 5 years ago

Last modified 9 days ago

#4209 open defect

GPS coordinates location and other iOS metadata in MOV are not copied to output MP4

Reported by: elmimmo Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: mov
Cc: tvolkert, aviram, genuinefafa@gmail.com Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug:

Certain metadata in MOV files recorded with iOS devices, such as GPS location, is lost when converting to MP4.

ffmpeg does state in its log that the output file will contain such metadata, but such is not the case as identified with ffprobe and exiftool.

Some of the metadata present in the input MOV that is missing in the output MP4, according to ffprobe:

  • make ("Apple")
  • encoder
  • location (GPS latitude/longitude/altitude coordinates)
  • model (the iPhone/iPad/iPod model)

exiftool identifies some more:

  • camera identifier ("Back" or "Front")
  • Software Version (iOS version)

This happens irrespective of input video/audio being transcoded or just repackaged.

How to reproduce:

% ffmpeg -i INPUT.mov -map_metadata 0 -c copy OUTPUT.mp4
ffmpeg version 2.5.2 Copyright (c) 2000-2014 the FFmpeg developers
  built on Dec 26 2014 21:55:58 with Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/2.5.2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libmp3lame --enable-libxvid --enable-libvo-aacenc --enable-vda
  libavutil      54. 15.100 / 54. 15.100
  libavcodec     56. 13.100 / 56. 13.100
  libavformat    56. 15.102 / 56. 15.102
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  2.103 /  5.  2.103
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'INPUT.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2014-12-26 20:54:22
    model           : iPhone 4
    model-spa       : iPhone 4
    encoder         : 7.1.1
    encoder-spa     : 7.1.1
    date            : 2014-12-26T21:54:22+0100
    date-spa        : 2014-12-26T21:54:22+0100
    location        : +41.4082+002.1852+027.119/
    location-spa    : +41.4082+002.1852+027.119/
    make            : Apple
    make-spa        : Apple
  Duration: 00:00:01.07, start: 0.000000, bitrate: 11694 kb/s
    Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720, 11890 kb/s, 24 fps, 24 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      rotate          : 90
      creation_time   : 2014-12-26 20:54:22
      handler_name    : Core Media Data Handler
      encoder         : H.264
    Side data:
      displaymatrix: rotation of -90.00 degrees
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 62 kb/s (default)
    Metadata:
      creation_time   : 2014-12-26 20:54:22
      handler_name    : Core Media Data Handler
Output #0, mp4, to 'OUTPUT.mp4':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2014-12-26 20:54:22
    model           : iPhone 4
    model-spa       : iPhone 4
    make-spa        : Apple
    encoder         : Lavf56.15.102
    date            : 2014-12-26T21:54:22+0100
    date-spa        : 2014-12-26T21:54:22+0100
    location        : +41.4082+002.1852+027.119/
    location-spa    : +41.4082+002.1852+027.119/
    make            : Apple
    Stream #0:0(und): Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 1280x720, q=2-31, 11890 kb/s, 24 fps, 19200 tbn, 600 tbc (default)
    Metadata:
      rotate          : 90
      creation_time   : 2014-12-26 20:54:22
      handler_name    : Core Media Data Handler
      encoder         : H.264
    Side data:
      displaymatrix: rotation of -90.00 degrees
    Stream #0:1(und): Audio: aac ([64][0][0][0] / 0x0040), 44100 Hz, mono, 62 kb/s (default)
    Metadata:
      creation_time   : 2014-12-26 20:54:22
      handler_name    : Core Media Data Handler
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=   25 fps=0.0 q=-1.0 Lsize=    1522kB time=00:00:01.06 bitrate=11673.4kbits/s    
video:1512kB audio:8kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.123020%

Attachments (8)

ffprobe.INPUT.mov.log (2.0 KB) - added by elmimmo 5 years ago.
ffprobe output of INPUT.mov
ffprobe.OUTPUT.mp4.log (1.7 KB) - added by elmimmo 5 years ago.
ffprobe output of OUTPUT.mp4
exiftool.INPUT.mov.log (4.2 KB) - added by elmimmo 5 years ago.
exiftool output of INPUT.mov
exiftool.OUTPUT.mp4.log (2.8 KB) - added by elmimmo 5 years ago.
exiftool output of OUTPUT.mp4
INPUT.mov (1.5 MB) - added by elmimmo 5 years ago.
exiftool-output.json (2.9 KB) - added by aviram 4 years ago.
Output of 'exiftool -j' on a video file recorded on an iPhone
has-location.mp4 (1.7 MB) - added by tvolkert 4 years ago.
Nexus 6 sample video with location info attached
0001-Add-xyz-atom-for-3gp-and-mp4-files-not-just-mov-file.patch (1.3 KB) - added by tvolkert 4 years ago.

Change History (30)

Changed 5 years ago by elmimmo

ffprobe output of INPUT.mov

Changed 5 years ago by elmimmo

ffprobe output of OUTPUT.mp4

Changed 5 years ago by elmimmo

exiftool output of INPUT.mov

Changed 5 years ago by elmimmo

exiftool output of OUTPUT.mp4

Changed 5 years ago by elmimmo

comment:1 Changed 5 years ago by compn

  • Component changed from undetermined to ffmpeg
  • Reproduced by developer set
  • Version changed from unspecified to git-master

ffmpeg does copy over the individual jpeg metadata
when you use ffmpeg -i file.jpg -c copy new.jpg

it appears ffmpeg output is lying when it says it will copy the metadata.

looks like rotation is copied, create date is copied.

comment:2 Changed 4 years ago by aviram

Any chance for this to be fixed ?

Tags that are missing on videos are Create Date, GPS-related tags, video orientation, manufacturer and model, camera used (front or back).

Last edited 4 years ago by aviram (previous) (diff)

Changed 4 years ago by aviram

Output of 'exiftool -j' on a video file recorded on an iPhone

comment:3 Changed 4 years ago by tvolkert

Another vote for this getting fixed!

comment:4 Changed 4 years ago by cehoyos

  • Cc tvolkert aviram added
  • Component changed from ffmpeg to avformat
  • Keywords mov added

Is there any indication that the location ("\251xyz") atom is supported in isom (mp4)?
Do you have a mp4 file that shows location?

comment:5 Changed 4 years ago by tvolkert

I'll try to attach an mp4 that has location that was taken with my Nexus 6 phone (file upload just failed, but I'll try again). My old Moto X produced similar mp4s. ffprobe produces the following:

Metadata:

major_brand : mp42
minor_version : 0
compatible_brands: isommp42
creation_time : 2015-07-22 15:30:16
location : +40.7920-73.8864/
location-eng : +40.7920-73.8864/

Duration: 00:00:01.05, start: 0.000000, bitrate: 14029 kb/s

Stream #0:0(eng): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p, 1280x720, 11705 kb/s, SAR 1:1 DAR 16:9, 29.99 fps, 30 tbr, 90k tbn, 180k tbc (default)
Metadata:

creation_time : 2015-07-22 15:30:16
handler_name : VideoHandle?

Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 96 kb/s (default)
Metadata:

creation_time : 2015-07-22 15:30:16
handler_name : SoundHandle?

Note that that appears to *not* be using the "\251xyz" atom, but instead the "location" and "location-eng" atoms...

I also found other code that handles the extraction of the "\251xyz" atom, in case that validates its existence... https://code.google.com/p/android-source-browsing/source/browse/media/libstagefright/MPEG4Extractor.cpp?repo=platform--frameworks--base&name=ics-mr1&r=4d7bc65538c7cd9fbb1fbbcf22d1da47fcee1219#1141

Changed 4 years ago by tvolkert

Nexus 6 sample video with location info attached

comment:6 Changed 4 years ago by tvolkert

Would there be any issue just passing through any metadata tags (including custom tags) found when using the -map_metadata? Then if the caller put invalid metadata tags in there and produced an unsupported mp4 file, it's their error (not the library's), and we don't have to enumerate the supported tags in code.

comment:7 Changed 4 years ago by cehoyos

The existing comments in movenc.c indicate that some applications have issues with random tags in isom files but imo feel free to send a patch.

comment:8 Changed 4 years ago by aviram

Here is a video recorded on my iPhone 5S that has geolocation embedded: http://avi.alkalay.net/clipboard/iPhone5S.m4v

I want to add more tracks to the video while cloning it and not losing the metadata:

ffmpeg -i iPhone5S.m4v -acodec copy -vcodec copy {parameters to add subtitles} iPhone5S.subtitled.m4v

Metadata would be maker and model, geo, orientation etc cited above.

Last edited 4 years ago by aviram (previous) (diff)

comment:9 Changed 4 years ago by cehoyos

  • Status changed from new to open

comment:10 Changed 4 years ago by tvolkert

Just attached a patch that adds the location ("\251xyz") atom in mp4. I've tested it by attaching the atom to a video and verifying that it plays on iOS, iTunes, Android video player, YouTube?, QuickTime?, and Google Photos.

comment:11 Changed 4 years ago by llogan

Patches on the bug tracker are often forgotten and is not suitable for proper reviews. Patches should be sent to the ffmpeg-devel mailing list:

http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

comment:12 Changed 4 years ago by tvolkert

Thanks - submitted the patch to ffmpeg-devel.

comment:13 Changed 2 years ago by Sendy

Hi,
sadly this does not meet the "other iOS metadata" part of the bug description.
I also want to stress that this does concern ALL metadata tags from other devices / operating systems as well (e.g. the "com.android.version" on android devices which should also be kept for reference)!

comment:14 Changed 15 months ago by genuinefafa

  • Cc genuinefafa@gmail.com added

I found that none of the Quicktime metadata is being copied either, losing GPS (and such) is not a good way to encode for archiving... :-/

comment:15 follow-up: Changed 10 months ago by Gyan

Adding -movflags use_metadata_tags will allow you to add and carry over custom metadata. ff* tools, at least, will show these tags in the output.

comment:16 in reply to: ↑ 15 Changed 10 months ago by genuinefafa

Replying to Gyan:

Adding -movflags use_metadata_tags will allow you to add and carry over custom metadata. ff* tools, at least, will show these tags in the output.

I just tested
%ffmpeg% -benchmark -i %input% -movflags use_metadata_tags -map_metadata 0:g -c:v libx264 -crf %video-encode% -c:a aac %output%

%ffmpeg% points to ffmpeg static 4.0 on Windows 10

it did not copy

com.apple.quicktime.location.I : +40.6195-074.1643+005.201/
com.apple.quicktime.make       : Apple
com.apple.quicktime.model      : iPhone 7
com.apple.quicktime.software   : 12.1.1
com.apple.quicktime.creationda : 2018-11-21T16:47:56-0500

into output.

Last edited 10 months ago by genuinefafa (previous) (diff)

comment:17 Changed 10 months ago by Gyan

Works here with INPUT.mov attached above. Share full log and input file.

comment:18 Changed 10 months ago by genuinefafa

Well, I did some testing, no trouble by using ffprobe. It does show as it should (information from source is copied to destination);

what I don't understand is why mpc-hc (media player home cinema) does show in file->properties->mediainfo with the source .mov and not the created .mp4. Is it possible that ffmpeg is not sticking to specifications and ffprobe either?

It might be a mpc-hc issue... :-/

comment:19 Changed 10 months ago by Gyan

The muxer, when adding these non-standard metadata, does so under a child mdta box which these other apps bypass. I'll look into writing it at top-level udta, when I can.

comment:20 Changed 10 months ago by genuinefafa

cheers to that Gyan! :D

comment:21 Changed 10 months ago by aviram

I just tested with ffmpeg version 4.0.3 with an iPhone X HEVC video.
Command was simply:

ffmpeg -i IMG_9988.MOV -movflags use_metadata_tags -c copy aaaa.mov

I can see the original metadata on target file (GPS, model etc) with ffprobe:

    com.apple.quicktime.creationdate: 2018-11-28T17:47:53-0200
    com.apple.quicktime.location.ISO6709: -23.5424-046.6576/
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone X
    com.apple.quicktime.software: 12.1

But Apple stack doesn't show metadata on target file. Here is a screenshot showing on left the source and target on right:

https://avi.alkalay.net/clipboard/ffmpeg-bug-4209.png

I can provide sample video files if needed.

comment:22 Changed 9 days ago by kvasa

This seems to be the issue - when you load the input and output files using AVFoundation (as AVAsset) and read the metadata, here is the difference:

Input:

   [...]
    <AVMetadataItem: 0x7fb602ef1ce0, identifier=mdta/com.apple.quicktime.location.ISO6709, keySpace=mdta, key class = __NSCFString, key=com.apple.quicktime.location.ISO6709, commonKey=location, extendedLanguageTag=(null), dataType=com.apple.metadata.datatype.UTF-8, time={INVALID}, duration={INVALID}, startDate=(null), extras={
    dataType = 1;
    dataTypeNamespace = "com.apple.quicktime.mdta";
}, value class=__NSCFString, value=+XX.6405-YY.8118+371.450/>
   [...]

Output file:

   [...]
   <AVMetadataItem: 0x7ff2d5217a50, identifier=itsk/%00%00%00%05, keySpace=itsk, key class = __NSCFNumber, key=, commonKey=(null), extendedLanguageTag=(null), dataType=com.apple.metadata.datatype.UTF-8, time={INVALID}, duration={INVALID}, startDate=(null), extras={
    dataType = 1;
    dataTypeNamespace = "com.apple.itunes";
}, value class=__NSCFString, value=+XX.6405-YY.8118+371.450/>
   [...]

Key differences:

  • identifier is set to itsk/%00%00%00%05 (as it's the fifth metadata item), instead of copying the input identifier mdta/com.apple.quicktime.location.ISO6709
  • namespace is set to com.apple.itunes instead of com.apple.quicktime.mdta

It is very likely that Apple requires the metadata items to be in the correct namespace, with the right identifier. I am not sure what is the exact issue as searching through FFmpeg source code, I can't find itsk or com.apple.itunes, so I am assuming that these get generated by AVFoundation if the correct value cannot be read...

Note: See TracTickets for help on using tickets.