Remuxed XDCAM mov does not play correctly in Quicktime player

[rmk]

I have remuxed an original XDCAM file created by an FCP7 export and the resulting file is not played back correctly by Quicktime. It skips the first two frames when played back and when pausing at the file's beginning and then stepping ahead frame by frame it starts at frame 6 (counting from 0) but then it is possible to step back until frame 2.

Command line and output:

ffmpeg -i /Users/krueger/tmp/playground/transcoding/fcp7_xdcam_hd422_1080_25p_syncbeep_2s.mov -vcodec copy -acodec copy /Users/krueger/tmp/playground/fcp7_xdcam_hd422_1080_25p_syncbeep_2s_ffmpeg_head.mov 
ffmpeg version N-53057-gbc1c90e Copyright (c) 2000-2013 the FFmpeg developers
  built on May 15 2013 16:34:42 with llvm-gcc 4.2.1 (LLVM build 2336.11.00)
  configuration: --disable-everything --enable-muxer=mov --enable-demuxer=mov --enable-protocol=file
  libavutil      52. 31.100 / 52. 31.100
  libavcodec     55.  9.100 / 55.  9.100
  libavformat    55.  7.100 / 55.  7.100
  libavdevice    55.  0.100 / 55.  0.100
  libavfilter     3. 67.100 /  3. 67.100
  libswscale      2.  3.100 /  2.  3.100
  libswresample   0. 17.102 /  0. 17.102
Guessed Channel Layout for  Input Stream #0.1 : stereo
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/krueger/tmp/playground/transcoding/fcp7_xdcam_hd422_1080_25p_syncbeep_2s.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 537199360
    compatible_brands: qt  
    creation_time   : 2013-05-14 10:36:21
  Duration: 00:00:02.00, start: 0.000000, bitrate: 52334 kb/s
    Stream #0:0(eng): Video: mpeg2video (xd5e / 0x65356478), 1920x1080, 49994 kb/s, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 25 tbn, 25 tbc
    Metadata:
      creation_time   : 2013-05-14 10:36:21
      handler_name    : Apple Alias-Datensteuerung
      timecode        : 01:00:00:00
    Stream #0:1(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, stereo, 2304 kb/s
    Metadata:
      creation_time   : 2013-05-14 10:36:21
      handler_name    : Apple Alias-Datensteuerung
    Stream #0:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s
    Metadata:
      creation_time   : 2013-05-14 10:36:23
      handler_name    : Apple Alias-Datensteuerung
      timecode        : 01:00:00:00
File '/Users/krueger/tmp/playground/fcp7_xdcam_hd422_1080_25p_syncbeep_2s_ffmpeg_head.mov' already exists. Overwrite ? [y/N] y
Output #0, mov, to '/Users/krueger/tmp/playground/fcp7_xdcam_hd422_1080_25p_syncbeep_2s_ffmpeg_head.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 537199360
    compatible_brands: qt  
    encoder         : Lavf55.7.100
    Stream #0:0(eng): Video: mpeg2video (xd5e / 0x65356478), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 49994 kb/s, 25 fps, 12800 tbn, 25 tbc
    Metadata:
      creation_time   : 2013-05-14 10:36:21
      handler_name    : Apple Alias-Datensteuerung
      timecode        : 01:00:00:00
    Stream #0:1(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, stereo, 2304 kb/s
    Metadata:
      creation_time   : 2013-05-14 10:36:21
      handler_name    : Apple Alias-Datensteuerung
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=   50 fps=0.0 q=-1.0 Lsize=   12771kB time=00:00:02.00 bitrate=52310.2kbits/s    
video:12206kB audio:562kB subtitle:0 global headers:0kB muxing overhead 0.022869%

[rmk]

I uploaded the file (fcp7_xdcam_hd422_1080_25p_syncbeep_2s.mov) to incoming

[rmk]

It is your project but If not being able to play back the file correctly on the de-facto reference player for this format is not a defect, then what is? Not that it matters much, though, so peace! No offense intended.

[rmk]

The edit list for the video track in the output file definitely looks broken. If I suppress edit list generation by specifying "-use_editlist 0" on the command line, the behaviour changes but is still broken. Now Quicktime playback starts with frame number 2 and I can step back until frame number 0. It may be a coincidence that it is skipping the first two frames but IMHO it's not too unlikely it's related with Ticket 479 as I don't think anything has changed about ffmpeg's handling of files starting with B-Frames since I posted the bug report.

[Carl Eugen Hoyos]

Probably a duplicate of ticket #1600.

[nele]

This thread helped me solve something I was working on, so here is what I did that you can duplicate in your code. The problem here is that frames here get packed so that you have I frame at the beginning, then two B frames and so on. The I frame's display time is actually shifted to the future so the B frame just behind it has to be displayed first. If you set the media_time to that frame's time, it should be just fine. For example, if I have 29.97 video, and my frames times are at 0/30000, 1001/30000, 2002/30000 and so on, the media_time has to be set to 1001. Hope this helps. Cheers.

[Carl Eugen Hoyos]

Is this still reproducible with current git head?

[rmk]

The same command line now produces a file which plays fine in Quicktime and FCP7. So I would consider this fixed. Thanks!

[Carl Eugen Hoyos]

Thank you for testing!
Probably fixed in e4d45673

[rmk]

Just for documentation purposes: It is not the codec tag that fixed it (i.e. e4d45673). We initially ran into the problem when using the muxer programatically and there we set the tag correctly already. I just made the command line example to report the bug. It must rather be a change to the handling of the ctts entries that used to be incorrect and now are correct, so it looks as if one of the mov commits merged over from libav fixed it (something like be401448e51f209a13e95d30f17d85d0168e1b40 but I did not check if that was the one fixing it).

[rmk]

I have to apologize. The bug is still there as it used to be. I accidentally used a locally patched version that has a hack that fixes it. So it has to be reopened. Sorry for wasting your time.

[Carl Eugen Hoyos]

Replying to rmk:

I have to apologize. The bug is still there as it used to be. I accidentally used a locally patched version that has a hack that fixes it.

Did you attach the hack here?

[rmk]

No but will do after it has been rebased to current head and cleaned up a bit.

[rmk]

Hack that fixes the behaviour for this case

[rmk]

Patch by Michael Luckey. Hope it helps.

[rmk]

Explanation of the patch:

Changes in mov_write_edts

The current implementation assumes, that a track is delayed by the cts of the first decoded frame.
(frame.dts + frame.cts = frame.pts) and inserts an empty edit into the edts atom if frame.cts > 0.

In case of xdcam, the stream starts with two B-frames, the stream is reordered and the first decoded frame, but not the track, will be delayed by these 2 frames.

So this assumption is certainly wrong in case of xdcam.

For example, lets assume the track starts with IBBPBB in decoding order, which results in BBIBBP in presentation order, we might have

I dts = 0, cts = 2 -> pts = 2
P dts = 1, cts = -1 -> pts = 0
P dts = 2, cts = -1 -> pts = 1
P dts = 3, cts = 2 -> pts = 5
P dts = 4, cts = -1 -> pts = 3
P dts = 5, cts = -1 -> pts = 4

the current implementation will insert an empty edit entry of the form

80, -1, 1

which will result in an edit-list-aware player starting playback at the frame with pts = 2. The track will be delayed by these 2 frames in respect to the container start. (This likely will also result in avsync issues?)

Contrary to that, the patch searches for the smallest pts available in the track. Only if this is > 0, an empty edit will be inserted and the track delayed.

With this patch applied, the problem in ​http://ffmpeg.org/trac/ffmpeg/ticket/2573 seems to be fixed.

Changes in mov_write_ctts

But looking at the ctts atom, there seems to be a second problem, as the computation does not take into account the decoding-presentation delay introduced by the reordering (in this example 1 frame). Thus the current implementation would issue a ctts atom of

e.g.

1, 3 (pts = 3)
2, 0 (pts = 1, pts = 2)
1, 3 (pts = 6)

instead of

1,2 (pts = 2)
2,-1 (pts = 0, pts = 1)
1,2 (pts = 5)

resulting in pts of the first shown frame being 1 instead of 0. The second part of the patch affecting mov_write_ctts tries to take this into account.