Timecode from XDCAM-EX MP4 file

[RCAB]

Summary of the bug: ffmpeg and ffprobe are reporting TIMECODE:00:00:00:00 while the XDCAM Browser is showing 02:52:22:04
File has Embedded XML at the end with the correct TimeCode (written backwards) "04225202"
<NonRealTimeMeta xmlns="urn:schemas-professionalDisc:nonRealTimeMeta:ver.1.20" xmlns:lib="urn:schemas-professionalDisc:lib:ver.1.20" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" lastUpdate="2009-12-01T14:53:49Z">

<TargetMaterial umidRef="060A2B340101010501010D43130000007CA5DE9E665105DC0800460202372A0F"/>
<Duration value="246"/>
<LtcChangeTable tcFps="25" halfStep="false">

<LtcChange frameCount="0" value="04225202" status="increment"/>
<LtcChange frameCount="245" value="24315202" status="end"/>

</LtcChangeTable>
<CreationDate value="2009-12-01T14:53:38Z"/>
<VideoFormat>

<VideoRecPort port="DIRECT"/>
<VideoFrame videoCodec="MPEG2HD35_1920_1080_MP@HL" captureFps="25.00p" recFps="25.00" formatFps="25p"/>
<VideoLayout pixel="1920" numOfVerticalLine="1080" aspectRatio="16:9"/>

</VideoFormat>
<AudioFormat numOfChannel="2">

<AudioRecPort port="ANALOG" audioCodec="LPCM16" trackDst="CH1"/>
<AudioRecPort port="DIRECT" audioCodec="LPCM16" trackDst="CH2"/>

</AudioFormat>
<RecordingMode type="normal" cacheRec="false"/>

</NonRealTimeMeta>

How to reproduce:

% ffmpeg -i XDCAM-EX-Sample-With_embedded_XML.MP4
% ffprobe -show_streams XDCAM-EX-Sample-With_embedded_XML.MP4

Result:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'XDCAM-EX-Sample-With_embedded_XML.MP4':

Metadata:

major_brand : mp42
minor_version : 0
compatible_brands: mp42
creation_time : 2009-12-01 14:53:38

Duration: 00:00:09.84, start: 0.000000, bitrate: 35761 kb/s

Stream #0:0(eng): Video: mpeg2video (Main) (mp4v / 0x7634706D), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 34220 kb/s, 2

5 fps, 25 tbr, 25k tbn, 50 tbc

Metadata:

creation_time : 2009-12-01 14:53:38
handler_name : Video Media Handler

Stream #0:1(eng): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, 2 channels, s16, 1536 kb/s
Metadata:

creation_time : 2009-12-01 14:53:38
handler_name : NRT Metadata Handler

[STREAM]
index=0
codec_name=mpeg2video
codec_long_name=MPEG-2 video
profile=Main
codec_type=video
codec_time_base=1/50
codec_tag_string=mp4v
codec_tag=0x7634706d
width=1920
height=1080
has_b_frames=1
sample_aspect_ratio=1:1
display_aspect_ratio=16:9
pix_fmt=yuv420p
level=4
timecode=00:00:00:00
id=N/A
r_frame_rate=25/1
avg_frame_rate=25/1
time_base=1/25000
start_pts=3000
start_time=0.120000
duration_ts=246000
duration=9.840000
bit_rate=34220903
nb_frames=246
nb_read_frames=N/A
nb_read_packets=N/A
DISPOSITION:default=0
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
TAG:creation_time=2009-12-01 14:53:38
TAG:language=eng
TAG:handler_name=Video Media Handler
STREAM
[STREAM]
index=1
codec_name=pcm_s16be
codec_long_name=PCM signed 16-bit big-endian
profile=unknown
codec_type=audio
codec_time_base=1/48000
codec_tag_string=twos
codec_tag=0x736f7774
sample_fmt=s16
sample_rate=48000
channels=2
bits_per_sample=16
id=N/A
r_frame_rate=0/0
avg_frame_rate=0/0
time_base=1/48000
start_pts=0
start_time=0.000000
duration_ts=472320
duration=9.840000
bit_rate=1536000
nb_frames=472320
nb_read_frames=N/A
nb_read_packets=N/A
DISPOSITION:default=0
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
TAG:creation_time=2009-12-01 14:53:38
TAG:language=eng
TAG:handler_name=NRT Metadata Handler
STREAM

[RCAB]

File is 45MB is to big to upload...
If needed i can put it on FTP

[Carl Eugen Hoyos]

Is this only reproducible with 1.1.1 or also with current git head?

Tickets without a sample are less likely to get fixed than tickets with a sample. Either use ​http://www.datafilehost.com/ or consider reading ​http://ffmpeg.org/bugreports.html

[Carl Eugen Hoyos]

And to make this a valid ticket, please provide the complete, uncut console output of a failing command lines (preferably the ffmpeg command line), please never provide an excerpt of the console output.

[RCAB]

Sorry for the trouble I am new in this :)

File is uploaded ​http://www.datafilehost.com/download-98df9c4d.html

---

complete output of ffprobe:

C:\FFMPEG64\bin>ffprobe -show_streams XDCAM-EX-Sample-With_embedded_XML.MP4
ffprobe version N-49069-gc74d465 Copyright (c) 2007-2013 the FFmpeg developers
  built on Jan 20 2013 23:44:08 with gcc 4.7.2 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib --enable-filter=frei0r
  libavutil      52. 15.100 / 52. 15.100
  libavcodec     54. 89.100 / 54. 89.100
  libavformat    54. 61.101 / 54. 61.101
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 32.101 /  3. 32.101
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'XDCAM-EX-Sample-With_embedded_XML.MP4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42
    creation_time   : 2009-12-01 14:53:38
  Duration: 00:00:09.84, start: 0.000000, bitrate: 35761 kb/s
    Stream #0:0(eng): Video: mpeg2video (Main) (mp4v / 0x7634706D), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 34220 kb/s, 25 fps, 25 tbr, 25k tbn, 50 tbc
    Metadata:
      creation_time   : 2009-12-01 14:53:38
      handler_name    : Video Media Handler
    Stream #0:1(eng): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, 2 channels, s16, 1536 kb/s
    Metadata:
      creation_time   : 2009-12-01 14:53:38
      handler_name    : NRT Metadata Handler
[STREAM]
index=0
codec_name=mpeg2video
codec_long_name=MPEG-2 video
profile=Main
codec_type=video
codec_time_base=1/50
codec_tag_string=mp4v
codec_tag=0x7634706d
width=1920
height=1080
has_b_frames=1
sample_aspect_ratio=1:1
display_aspect_ratio=16:9
pix_fmt=yuv420p
level=4
timecode=00:00:00:00
id=N/A
r_frame_rate=25/1
avg_frame_rate=25/1
time_base=1/25000
start_pts=3000
start_time=0.120000
duration_ts=246000
duration=9.840000
bit_rate=34220903
nb_frames=246
nb_read_frames=N/A
nb_read_packets=N/A
DISPOSITION:default=0
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
TAG:creation_time=2009-12-01 14:53:38
TAG:language=eng
TAG:handler_name=Video Media Handler
[/STREAM]
[STREAM]
index=1
codec_name=pcm_s16be
codec_long_name=PCM signed 16-bit big-endian
profile=unknown
codec_type=audio
codec_time_base=1/48000
codec_tag_string=twos
codec_tag=0x736f7774
sample_fmt=s16
sample_rate=48000
channels=2
bits_per_sample=16
id=N/A
r_frame_rate=0/0
avg_frame_rate=0/0
time_base=1/48000
start_pts=0
start_time=0.000000
duration_ts=472320
duration=9.840000
bit_rate=1536000
nb_frames=472320
nb_read_frames=N/A
nb_read_packets=N/A
DISPOSITION:default=0
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
TAG:creation_time=2009-12-01 14:53:38
TAG:language=eng
TAG:handler_name=NRT Metadata Handler
[/STREAM]

[Stefano Sabatini]

Replying to RCAB:

Summary of the bug: ffmpeg and ffprobe are reporting TIMECODE:00:00:00:00 while the XDCAM Browser is showing 02:52:22:04
File has Embedded XML at the end with the correct TimeCode (written backwards) "04225202"

[...]

This is the first time I see embedded XML at the end of a stream. Is this supposed to be a standard format? If yes where is it documented?

[RCAB]

I have 1 Disc with 8 clips for testing XDCAM EX files with my scripts which transcode by using ffmpeg the files into lowres quicktime format but I also need the TIMECODE for use in AVID.
I use ffprobe to analyze the incoming files.
All videofiles have a seperate XML file.
But the basic part of the XML file is also embedded in the MP4 files, always at the end of the file.
ffprobe is now telling me that the TIMECODE: 00:00:00:00
It would be nice if I could read the TIMECODE directly from the MP4 file instead of the seperate XML file.

[Stefano Sabatini]

Replying to RCAB:

I have 1 Disc with 8 clips for testing XDCAM EX files with my scripts which transcode by using ffmpeg the files into lowres quicktime format but I also need the TIMECODE for use in AVID.
I use ffprobe to analyze the incoming files.
All videofiles have a seperate XML file.
But the basic part of the XML file is also embedded in the MP4 files, always at the end of the file.
ffprobe is now telling me that the TIMECODE: 00:00:00:00
It would be nice if I could read the TIMECODE directly from the MP4 file instead of the seperate XML file.

I see but what I'm asking is if you're aware of any spec document dictating how such metadata should be retrieved, or if this is specific to the application you're using to create and read the file (in other words, why is the program not using the usual way to mark metadata?). That said, it would be in theory possible to read the information from the file even if highly unpractical since it is stored at the end of the file, and would require proper XML parsing (and thus require a dependency on an XML parsing library).

[RCAB]

I am not aware of any documentation.
During my testing with this files I found that when I use HexEditor to change the XML at the end of the file and save the file. The Sony XDCAM browser v2.1 is showing the changed values.
But this only happens when I play it without the seperate XML file.
If the original XML file is still there it takes the values from the XML files instead of the modified values in the embedded XML.
My testfiles are only 40... 400MB (9..90 seconds) but real files van be very big files. So I see your point in being unpractical to read the ..GB files to find the XML structure.