Opened 9 years ago

Last modified 12 months ago

#4209 open defect

GPS coordinates location and other iOS metadata in MOV are not copied to output MP4

Reported by: elmimmo Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: mov
Cc: tvolkert, Avi Alkalay, genuinefafa@gmail.com, tim.ffmpeg@vanwerkhoven.org Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug:

Certain metadata in MOV files recorded with iOS devices, such as GPS location, is lost when converting to MP4.

ffmpeg does state in its log that the output file will contain such metadata, but such is not the case as identified with ffprobe and exiftool.

Some of the metadata present in the input MOV that is missing in the output MP4, according to ffprobe:

  • make ("Apple")
  • encoder
  • location (GPS latitude/longitude/altitude coordinates)
  • model (the iPhone/iPad/iPod model)

exiftool identifies some more:

  • camera identifier ("Back" or "Front")
  • Software Version (iOS version)

This happens irrespective of input video/audio being transcoded or just repackaged.

How to reproduce:

% ffmpeg -i INPUT.mov -map_metadata 0 -c copy OUTPUT.mp4
ffmpeg version 2.5.2 Copyright (c) 2000-2014 the FFmpeg developers
  built on Dec 26 2014 21:55:58 with Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/2.5.2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libmp3lame --enable-libxvid --enable-libvo-aacenc --enable-vda
  libavutil      54. 15.100 / 54. 15.100
  libavcodec     56. 13.100 / 56. 13.100
  libavformat    56. 15.102 / 56. 15.102
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  2.103 /  5.  2.103
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'INPUT.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2014-12-26 20:54:22
    model           : iPhone 4
    model-spa       : iPhone 4
    encoder         : 7.1.1
    encoder-spa     : 7.1.1
    date            : 2014-12-26T21:54:22+0100
    date-spa        : 2014-12-26T21:54:22+0100
    location        : +41.4082+002.1852+027.119/
    location-spa    : +41.4082+002.1852+027.119/
    make            : Apple
    make-spa        : Apple
  Duration: 00:00:01.07, start: 0.000000, bitrate: 11694 kb/s
    Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720, 11890 kb/s, 24 fps, 24 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      rotate          : 90
      creation_time   : 2014-12-26 20:54:22
      handler_name    : Core Media Data Handler
      encoder         : H.264
    Side data:
      displaymatrix: rotation of -90.00 degrees
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 62 kb/s (default)
    Metadata:
      creation_time   : 2014-12-26 20:54:22
      handler_name    : Core Media Data Handler
Output #0, mp4, to 'OUTPUT.mp4':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2014-12-26 20:54:22
    model           : iPhone 4
    model-spa       : iPhone 4
    make-spa        : Apple
    encoder         : Lavf56.15.102
    date            : 2014-12-26T21:54:22+0100
    date-spa        : 2014-12-26T21:54:22+0100
    location        : +41.4082+002.1852+027.119/
    location-spa    : +41.4082+002.1852+027.119/
    make            : Apple
    Stream #0:0(und): Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 1280x720, q=2-31, 11890 kb/s, 24 fps, 19200 tbn, 600 tbc (default)
    Metadata:
      rotate          : 90
      creation_time   : 2014-12-26 20:54:22
      handler_name    : Core Media Data Handler
      encoder         : H.264
    Side data:
      displaymatrix: rotation of -90.00 degrees
    Stream #0:1(und): Audio: aac ([64][0][0][0] / 0x0040), 44100 Hz, mono, 62 kb/s (default)
    Metadata:
      creation_time   : 2014-12-26 20:54:22
      handler_name    : Core Media Data Handler
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=   25 fps=0.0 q=-1.0 Lsize=    1522kB time=00:00:01.06 bitrate=11673.4kbits/s    
video:1512kB audio:8kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.123020%

Attachments (8)

ffprobe.INPUT.mov.log (2.0 KB ) - added by elmimmo 9 years ago.
ffprobe output of INPUT.mov
ffprobe.OUTPUT.mp4.log (1.7 KB ) - added by elmimmo 9 years ago.
ffprobe output of OUTPUT.mp4
exiftool.INPUT.mov.log (4.2 KB ) - added by elmimmo 9 years ago.
exiftool output of INPUT.mov
exiftool.OUTPUT.mp4.log (2.8 KB ) - added by elmimmo 9 years ago.
exiftool output of OUTPUT.mp4
INPUT.mov (1.5 MB ) - added by elmimmo 9 years ago.
exiftool-output.json (2.9 KB ) - added by Avi Alkalay 9 years ago.
Output of 'exiftool -j' on a video file recorded on an iPhone
has-location.mp4 (1.7 MB ) - added by tvolkert 9 years ago.
Nexus 6 sample video with location info attached
0001-Add-xyz-atom-for-3gp-and-mp4-files-not-just-mov-file.patch (1.3 KB ) - added by tvolkert 8 years ago.

Change History (33)

by elmimmo, 9 years ago

Attachment: ffprobe.INPUT.mov.log added

ffprobe output of INPUT.mov

by elmimmo, 9 years ago

Attachment: ffprobe.OUTPUT.mp4.log added

ffprobe output of OUTPUT.mp4

by elmimmo, 9 years ago

Attachment: exiftool.INPUT.mov.log added

exiftool output of INPUT.mov

by elmimmo, 9 years ago

Attachment: exiftool.OUTPUT.mp4.log added

exiftool output of OUTPUT.mp4

by elmimmo, 9 years ago

Attachment: INPUT.mov added

comment:1 by compn, 9 years ago

Component: undeterminedffmpeg
Reproduced by developer: set
Version: unspecifiedgit-master

ffmpeg does copy over the individual jpeg metadata
when you use ffmpeg -i file.jpg -c copy new.jpg

it appears ffmpeg output is lying when it says it will copy the metadata.

looks like rotation is copied, create date is copied.

comment:2 by Avi Alkalay, 9 years ago

Any chance for this to be fixed ?

Tags that are missing on videos are Create Date, GPS-related tags, video orientation, manufacturer and model, camera used (front or back).

Last edited 9 years ago by Avi Alkalay (previous) (diff)

by Avi Alkalay, 9 years ago

Attachment: exiftool-output.json added

Output of 'exiftool -j' on a video file recorded on an iPhone

comment:3 by tvolkert, 9 years ago

Another vote for this getting fixed!

comment:4 by Carl Eugen Hoyos, 9 years ago

Cc: tvolkert Avi Alkalay added
Component: ffmpegavformat
Keywords: mov added

Is there any indication that the location ("\251xyz") atom is supported in isom (mp4)?
Do you have a mp4 file that shows location?

comment:5 by tvolkert, 9 years ago

I'll try to attach an mp4 that has location that was taken with my Nexus 6 phone (file upload just failed, but I'll try again). My old Moto X produced similar mp4s. ffprobe produces the following:

Metadata:

major_brand : mp42
minor_version : 0
compatible_brands: isommp42
creation_time : 2015-07-22 15:30:16
location : +40.7920-73.8864/
location-eng : +40.7920-73.8864/

Duration: 00:00:01.05, start: 0.000000, bitrate: 14029 kb/s

Stream #0:0(eng): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p, 1280x720, 11705 kb/s, SAR 1:1 DAR 16:9, 29.99 fps, 30 tbr, 90k tbn, 180k tbc (default)
Metadata:

creation_time : 2015-07-22 15:30:16
handler_name : VideoHandle

Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 96 kb/s (default)
Metadata:

creation_time : 2015-07-22 15:30:16
handler_name : SoundHandle

Note that that appears to *not* be using the "\251xyz" atom, but instead the "location" and "location-eng" atoms...

I also found other code that handles the extraction of the "\251xyz" atom, in case that validates its existence... https://code.google.com/p/android-source-browsing/source/browse/media/libstagefright/MPEG4Extractor.cpp?repo=platform--frameworks--base&name=ics-mr1&r=4d7bc65538c7cd9fbb1fbbcf22d1da47fcee1219#1141

by tvolkert, 9 years ago

Attachment: has-location.mp4 added

Nexus 6 sample video with location info attached

comment:6 by tvolkert, 9 years ago

Would there be any issue just passing through any metadata tags (including custom tags) found when using the -map_metadata? Then if the caller put invalid metadata tags in there and produced an unsupported mp4 file, it's their error (not the library's), and we don't have to enumerate the supported tags in code.

comment:7 by Carl Eugen Hoyos, 9 years ago

The existing comments in movenc.c indicate that some applications have issues with random tags in isom files but imo feel free to send a patch.

comment:8 by Avi Alkalay, 9 years ago

Here is a video recorded on my iPhone 5S that has geolocation embedded: http://avi.alkalay.net/clipboard/iPhone5S.m4v

I want to add more tracks to the video while cloning it and not losing the metadata:

ffmpeg -i iPhone5S.m4v -acodec copy -vcodec copy {parameters to add subtitles} iPhone5S.subtitled.m4v

Metadata would be maker and model, geo, orientation etc cited above.

Last edited 9 years ago by Avi Alkalay (previous) (diff)

comment:9 by Carl Eugen Hoyos, 9 years ago

Status: newopen

comment:10 by tvolkert, 8 years ago

Just attached a patch that adds the location ("\251xyz") atom in mp4. I've tested it by attaching the atom to a video and verifying that it plays on iOS, iTunes, Android video player, YouTube, QuickTime, and Google Photos.

comment:11 by llogan, 8 years ago

Patches on the bug tracker are often forgotten and is not suitable for proper reviews. Patches should be sent to the ffmpeg-devel mailing list:

http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

comment:12 by tvolkert, 8 years ago

Thanks - submitted the patch to ffmpeg-devel.

comment:13 by Sendy, 6 years ago

Hi,
sadly this does not meet the "other iOS metadata" part of the bug description.
I also want to stress that this does concern ALL metadata tags from other devices / operating systems as well (e.g. the "com.android.version" on android devices which should also be kept for reference)!

comment:14 by Facundo D, 6 years ago

Cc: genuinefafa@gmail.com added

I found that none of the Quicktime metadata is being copied either, losing GPS (and such) is not a good way to encode for archiving... :-/

comment:15 by Gyan, 5 years ago

Adding -movflags use_metadata_tags will allow you to add and carry over custom metadata. ff* tools, at least, will show these tags in the output.

in reply to:  15 comment:16 by Facundo D, 5 years ago

Replying to Gyan:

Adding -movflags use_metadata_tags will allow you to add and carry over custom metadata. ff* tools, at least, will show these tags in the output.

I just tested
%ffmpeg% -benchmark -i %input% -movflags use_metadata_tags -map_metadata 0:g -c:v libx264 -crf %video-encode% -c:a aac %output%

%ffmpeg% points to ffmpeg static 4.0 on Windows 10

it did not copy

com.apple.quicktime.location.I : +40.6195-074.1643+005.201/
com.apple.quicktime.make       : Apple
com.apple.quicktime.model      : iPhone 7
com.apple.quicktime.software   : 12.1.1
com.apple.quicktime.creationda : 2018-11-21T16:47:56-0500

into output.

Last edited 5 years ago by Facundo D (previous) (diff)

comment:17 by Gyan, 5 years ago

Works here with INPUT.mov attached above. Share full log and input file.

comment:18 by Facundo D, 5 years ago

Well, I did some testing, no trouble by using ffprobe. It does show as it should (information from source is copied to destination);

what I don't understand is why mpc-hc (media player home cinema) does show in file->properties->mediainfo with the source .mov and not the created .mp4. Is it possible that ffmpeg is not sticking to specifications and ffprobe either?

It might be a mpc-hc issue... :-/

comment:19 by Gyan, 5 years ago

The muxer, when adding these non-standard metadata, does so under a child mdta box which these other apps bypass. I'll look into writing it at top-level udta, when I can.

comment:20 by Facundo D, 5 years ago

cheers to that Gyan! :D

comment:21 by Avi Alkalay, 5 years ago

I just tested with ffmpeg version 4.0.3 with an iPhone X HEVC video.
Command was simply:

ffmpeg -i IMG_9988.MOV -movflags use_metadata_tags -c copy aaaa.mov

I can see the original metadata on target file (GPS, model etc) with ffprobe:

    com.apple.quicktime.creationdate: 2018-11-28T17:47:53-0200
    com.apple.quicktime.location.ISO6709: -23.5424-046.6576/
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone X
    com.apple.quicktime.software: 12.1

But Apple stack doesn't show metadata on target file. Here is a screenshot showing on left the source and target on right:

https://avi.alkalay.net/clipboard/ffmpeg-bug-4209.png

I can provide sample video files if needed.

comment:22 by kvasa, 4 years ago

This seems to be the issue - when you load the input and output files using AVFoundation (as AVAsset) and read the metadata, here is the difference:

Input:

   [...]
    <AVMetadataItem: 0x7fb602ef1ce0, identifier=mdta/com.apple.quicktime.location.ISO6709, keySpace=mdta, key class = __NSCFString, key=com.apple.quicktime.location.ISO6709, commonKey=location, extendedLanguageTag=(null), dataType=com.apple.metadata.datatype.UTF-8, time={INVALID}, duration={INVALID}, startDate=(null), extras={
    dataType = 1;
    dataTypeNamespace = "com.apple.quicktime.mdta";
}, value class=__NSCFString, value=+XX.6405-YY.8118+371.450/>
   [...]

Output file:

   [...]
   <AVMetadataItem: 0x7ff2d5217a50, identifier=itsk/%00%00%00%05, keySpace=itsk, key class = __NSCFNumber, key=, commonKey=(null), extendedLanguageTag=(null), dataType=com.apple.metadata.datatype.UTF-8, time={INVALID}, duration={INVALID}, startDate=(null), extras={
    dataType = 1;
    dataTypeNamespace = "com.apple.itunes";
}, value class=__NSCFString, value=+XX.6405-YY.8118+371.450/>
   [...]

Key differences:

  • identifier is set to itsk/%00%00%00%05 (as it's the fifth metadata item), instead of copying the input identifier mdta/com.apple.quicktime.location.ISO6709
  • namespace is set to com.apple.itunes instead of com.apple.quicktime.mdta

It is very likely that Apple requires the metadata items to be in the correct namespace, with the right identifier. I am not sure what is the exact issue as searching through FFmpeg source code, I can't find itsk or com.apple.itunes, so I am assuming that these get generated by AVFoundation if the correct value cannot be read...

comment:23 by TimvW1984, 4 years ago

Cc: tim.ffmpeg@vanwerkhoven.org added

Any updates on this? When inspecting the video with exiftool, in my case the hierarchy of metadata is different. Could this be it? iOS stores GPS in Movie.Meta while FFmpeg stores this in Movie.UserData.Meta.

Setup

ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
Exiftool version: 11.65

Original metadata

(GPS / datetime redacted)

exiftool -v <filename>
  Movie (SubDirectory) -->
  + [Movie directory]
  [...stuff...]
  | UserData (SubDirectory) -->
  | Unknown_free = 
  | Meta (SubDirectory) -->
  | + [Meta directory]
  | | Handler (SubDirectory) -->
  | | + [BinaryData directory, 26 bytes]
  | | | HandlerClass = 
  | | | HandlerType = mdta
  | | | HandlerVendorID = 
  | | | HandlerDescription = 
  | | Keys (SubDirectory) -->
  | | + [Keys directory]
  | | | Added ItemList Tag 1.1 = location.ISO6709
  | | | Added ItemList Tag 1.2 = make
  | | | Added ItemList Tag 1.3 = model
  | | | Added ItemList Tag 1.4 = software
  | | | Added ItemList Tag 1.5 = creationdate
  | | ItemList (SubDirectory) -->
  | | + [ItemList directory]
  | | | GPSCoordinates = +12.3456+001.2345+001.234/
  | | | Make = Apple
  | | | Model = iPhone 6s
  | | | Software = 13.1.3
  | | | CreationDate = 2019-11-02T01:23:45+0000
  | | Free = 
  | Unknown_free =  trex....... trex.... trex...v. trex...v.

Modified metadata

ffmpeg -i IMG_5585-orig.MOV -c:v libx264  -movflags use_metadata_tags -c:a libfdk_aac  IMG_5585-orig.MOV-x264_aac.mp4

(GPS / datetime redacted)

exiftool -v <filename>
  Movie (SubDirectory) -->
  + [Movie directory]
  [...stuff...]
  | UserData (SubDirectory) -->
  | + [UserData directory]
  | | Meta (SubDirectory) -->
  | | + [Meta directory]
  | | | Handler (SubDirectory) -->
  | | | + [BinaryData directory, 25 bytes]
  | | | | HandlerClass = 
  | | | | HandlerType = mdta
  | | | | HandlerVendorID = 
  | | | | HandlerDescription = 
  | | | Keys (SubDirectory) -->
  | | | + [Keys directory]
  | | | | Added ItemList Tag 1.1 = major_brand (Unknown)
  | | | | Added ItemList Tag 1.2 = minor_version (Unknown)
  | | | | Added ItemList Tag 1.3 = compatible_brands (Unknown)
  | | | | Added ItemList Tag 1.4 = creationdate
  | | | | Added ItemList Tag 1.5 = location.ISO6709
  | | | | Added ItemList Tag 1.6 = make
  | | | | Added ItemList Tag 1.7 = model
  | | | | Added ItemList Tag 1.8 = software
  | | | | Added ItemList Tag 1.9 = encoder (Unknown)
  | | | ItemList (SubDirectory) -->
  | | | + [ItemList directory]
  | | | | MajorBrand = qt  
  | | | | MinorVersion = 0
  | | | | CompatibleBrands = qt  
  | | | | CreationDate = 2019-11-02T01:23:45+0000
  | | | | GPSCoordinates = +12.3456+001.2345+001.234/
  | | | | Make = Apple
  | | | | Model = iPhone 6s
  | | | | Software = 13.1.3
  | | | | Encoder = Lavf58.29.100
Last edited 4 years ago by TimvW1984 (previous) (diff)

comment:24 by TimvW1984, 4 years ago

This issue is similar to https://trac.ffmpeg.org/ticket/6193, which also lists a manual work-around using mp4extract from https://www.bento4.com

Last edited 4 years ago by TimvW1984 (previous) (diff)

comment:25 by Oleksandr, 12 months ago

I faced the same issue as some users reported here: iOS doesn't show the location if media files were created by FFmpeg. After some analysis independently of TimvW1984, I've found where FFmpeg and iOS put their metadata tags:

FFmpeg: moov/udta/meta
iOS: moov/meta

so FFmpeg uses the improper location of the meta tag as already suggested by TimvW1984.

Furthermore, Apple's QuickTime specification [1] defines the correct location for the metadata tag:

Within a QuickTime file, metadata can be stored within a movie atom (‘moov’), a track atom (‘trak’) or a media atom (‘mdia’). Only one metadata atom is allowed for each location.

The current FFmpeg doesn't follow this specification and writes meta inside the udta atom[2].

From this, I can conclude: that is the cause why location metadata is not recognized by Apple's software.

[1]: https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/Metadata/Metadata.html

[2]: https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/refs/heads/release/5.0:/libavformat/movenc.c#l4054

Last edited 12 months ago by Oleksandr (previous) (diff)
Note: See TracTickets for help on using tickets.