Opened 20 months ago

Last modified 19 months ago

#5512 new defect

Audio Out of Sync after transcode

Reported by: ronag Owned by:
Priority: normal Component: undetermined
Version: git-master Keywords:
Cc: enevills Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

After transcoding the audio is off by 2 frames.

How to reproduce:

ronagy$ ffmpeg -i original.mov -flags +ilme+ildct transcoded.mov
ffmpeg version git-2016-05-05-5b174dd Copyright (c) 2000-2016 the FFmpeg developers
  built with Apple LLVM version 7.3.0 (clang-703.0.29)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-opencl --enable-libx264 --enable-libmp3lame --enable-libxvid --enable-libfreetype --enable-libvorbis --enable-libvpx --enable-librtmp --enable-libass --enable-ffplay --enable-libfdk-aac --enable-libopus --enable-libx265 --enable-nonfree --enable-vda
  libavutil      55. 23.100 / 55. 23.100
  libavcodec     57. 38.100 / 57. 38.100
  libavformat    57. 36.100 / 57. 36.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 44.100 /  6. 44.100
  libavresample   3.  0.  0 /  3.  0.  0
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'original.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf57.25.100
  Duration: 00:00:21.28, start: 0.000000, bitrate: 130459 kb/s
    Stream #0:0(eng): Video: dnxhd (AVdn / 0x6E645641), yuv422p(tv, bt709/unknown/unknown), 1920x1080, 121241 kb/s, 25 fps, 25 tbr, 12800 tbn (default)
    Metadata:
      handler_name    : DataHandler
      timecode        : 00:00:00:00
    Stream #0:1(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, 7.1, s32 (24 bit), 9216 kb/s (default)
    Metadata:
      handler_name    : DataHandler
    Stream #0:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s
    Metadata:
      handler_name    : DataHandler
      timecode        : 00:00:00:00
No pixel format specified, yuv422p for H.264 encoding chosen.
Use -pix_fmt yuv420p for compatibility with outdated media players.
[libx264 @ 0x7ff6b3811e00] interlace + weightp is not implemented
[libx264 @ 0x7ff6b3811e00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 0x7ff6b3811e00] profile High 4:2:2, level 4.0, 4:2:2 8-bit
[libx264 @ 0x7ff6b3811e00] 264 - core 148 r2601 a0cd7d3 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=tff bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=0 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[mov @ 0x7ff6b3810c00] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
    Last message repeated 1 times
Output #0, mov, to 'transcoded.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf57.36.100
    Stream #0:0(eng): Video: h264 (avc1 / 0x31637661), yuv422p, 1920x1080, q=2-31, 0.04 fps, 12800 tbn (default)
    Metadata:
      handler_name    : DataHandler
      timecode        : 00:00:00:00
      encoder         : Lavc57.38.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 7.1, fltp (24 bit), 469 kb/s (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc57.38.100 aac
Stream mapping:
  Stream #0:0 -> #0:0 (dnxhd (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (pcm_s24le (native) -> aac (native))
Press [q] to stop, [?] for help
frame=  532 fps=106 q=-1.0 Lsize=    1833kB time=00:00:21.29 bitrate= 705.2kbits/s speed=4.25x    
video:602kB audio:1213kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.012604%
[libx264 @ 0x7ff6b3811e00] frame I:3     Avg QP:17.24  size: 57340
[libx264 @ 0x7ff6b3811e00] frame P:143   Avg QP:24.25  size:  2370
[libx264 @ 0x7ff6b3811e00] frame B:386   Avg QP:31.72  size:   271
[libx264 @ 0x7ff6b3811e00] consecutive B-frames:  2.6%  1.5%  1.1% 94.7%
[libx264 @ 0x7ff6b3811e00] mb I  I16..4: 25.6% 66.1%  8.4%
[libx264 @ 0x7ff6b3811e00] mb P  I16..4:  0.3%  0.3%  0.1%  P16..4:  2.3%  0.3%  0.2%  0.0%  0.0%    skip:96.5%
[libx264 @ 0x7ff6b3811e00] mb B  I16..4:  0.0%  0.1%  0.0%  B16..8:  0.9%  0.1%  0.0%  direct: 0.0%  skip:98.9%  L0:28.6% L1:69.1% BI: 2.2%
[libx264 @ 0x7ff6b3811e00] field mbs: intra: 9.2% inter:6.5% skip:0.7%
[libx264 @ 0x7ff6b3811e00] 8x8 transform intra:63.2% inter:65.4%
[libx264 @ 0x7ff6b3811e00] coded y,uvDC,uvAC intra: 7.5% 12.5% 8.4% inter: 0.5% 0.1% 0.0%
[libx264 @ 0x7ff6b3811e00] i16 v,h,dc,p: 74% 24%  2%  0%
[libx264 @ 0x7ff6b3811e00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 40%  2% 56%  0%  0%  0%  0%  0%  0%
[libx264 @ 0x7ff6b3811e00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 36% 17% 25%  4%  4%  4%  4%  4%  3%
[libx264 @ 0x7ff6b3811e00] i8c dc,h,v,p: 68% 11% 20%  1%
[libx264 @ 0x7ff6b3811e00] ref P L0: 90.7%  4.1%  4.7%  0.0%  0.3%  0.1%
[libx264 @ 0x7ff6b3811e00] ref B L0: 66.9% 27.7%  5.1%  0.1%  0.2%  0.1%
[libx264 @ 0x7ff6b3811e00] ref B L1: 96.1%  3.6%  0.3%  0.0%
[libx264 @ 0x7ff6b3811e00] kb/s:231.39
[aac @ 0x7ff6b3813600] Qavg: 573.770

Files are available here:

https://www.dropbox.com/sh/9qi9813i3jm1mlz/AACkgGs5IP4AizOja0HfgtdUa?dl=0

Attachments (2)

original.png (53.9 KB) - added by ronag 20 months ago.
original sync
transcoded.png (58.8 KB) - added by ronag 20 months ago.
transcoded sync

Download all attachments as: .zip

Change History (26)

Changed 20 months ago by ronag

original sync

Changed 20 months ago by ronag

transcoded sync

comment:1 Changed 20 months ago by cehoyos

I believe this is a duplicate of one or more other tickets, I unfortunately cannot comment on these tickets as I have no idea how to reproduce them.

comment:2 Changed 20 months ago by ronag

@cehoyos: Why can't you reproduce this? I've provided the files and the command line. What are you missing?

comment:3 Changed 20 months ago by cehoyos

I am not saying I am missing something, I just don't know how to reproduce an off-by-two-frames. (I also believe the issue is not reproducible at all but that is not my point.)

Do you believe the issue is also reproducible with -vcodec mpeg4 -qscale 2?

comment:4 Changed 20 months ago by kierank

Probably crap b-frames handling in mov. Turn off bframes.
Kieran

comment:5 Changed 20 months ago by ronag

@Kieran: That did solve it.

comment:6 follow-up: Changed 20 months ago by ronag

Unfortunately it's all over the place. Especially when I involve a -ss infront of -i.

@cehoyos: What can I do to help finding a resolution to this issue?

comment:7 in reply to: ↑ 6 Changed 20 months ago by cehoyos

Replying to ronag:

What can I do to help finding a resolution to this issue?

Did you already test if you can reproduce the issue with -vcodec mpeg4 -qscale 2?
Is the issue more visiable (more than two frames off) if you use eight b-frames?

comment:8 Changed 20 months ago by ronag

If I run:

ffmpeg -ss 10.032 -i original.mov -flags +ilme+ildct -c:v mpeg4 -qscale:v 2 seeked2.mov

It's off by 34 frames.

comment:9 Changed 20 months ago by cehoyos

Is -flags +ilme+ildct needed to reproduce the issue?

comment:10 Changed 20 months ago by ronag

ffmpeg -ss 10.032 -i original.mov -flags +ilme+ildct seeked3.mov

Is off by 2 frames

ffmpeg -ss 10.032 -i original.mov -flags +ilme+ildct -bf:v 0 seeked4.mov

Is correct.

comment:11 Changed 20 months ago by ronag

@cehoyos: No. It's the same without the flags.

comment:12 Changed 20 months ago by ronag

With the following it's incorrect even when using kierans trick.

ffmpeg -ss 10.032 -i original.mxf -bf:v 0 seeked8.mov

Off by 1 frame.

comment:13 Changed 20 months ago by ronag

With the following it's incorrect even when using kierans trick.

ffmpeg -ss 10.032 -i original.mxf -bf:v 0 seeked8.mxf

Off by 1 frame.

Last edited 20 months ago by ronag (previous) (diff)

comment:14 Changed 20 months ago by cehoyos

I tested the following command line with current FFmpeg git head:

$ ffmpeg -ss 10.032 -i original.mov -qscale 2 -vcodec mpeg4 out.mov

I tested playback of the output file with the following two commands:

$ vlc --rate=0.25 out.mov
$ mplayer -speed 0.1 out.mov

Both do not show out-of-sync audio afaict.

comment:15 Changed 20 months ago by ronag

@cehoyos: Yes, you are correct. With that commandline it works. Not sure what I did wrong before.

Last edited 20 months ago by ronag (previous) (diff)

comment:16 Changed 20 months ago by ronag

@cehoyos:

However, using the file "3.mxf"

ffmpeg -ss 10.032 -i 3.mxf -qscale 2 -vcodec mpeg4 out10.mov

It's off by 1 frame.

comment:17 Changed 20 months ago by cehoyos

Can we keep one input file for testing please?

Is the effect accumulating? Does repeating the command line twenty times lead to twenty frames off?

Please understand that imo, there is no such thing as "off-by-one-frame" for real-world video, playback equipment (and speaker distance) is not identical for all users.

comment:18 Changed 20 months ago by ronag

Can we keep one input file for testing please?

Sure.

Please understand that imo, there is no such thing as "off-by-one-frame" for real-world video, playback equipment (and speaker distance) is not identical for all users.

I need the video and audio to be in sync since it's to be used in After Effects and AVID edit suites. The files are not directly used for playback.

comment:19 follow-up: Changed 19 months ago by enevills

I see the same thing when making x264 files with AAC audio. After Effects and Avid see files at 2 frames off.

Command:

ffmpeg -loglevel debug -i 'file.mov' -pix_fmt yuv420p -vf scale=960:540 -c:v libx264 -preset veryslow -crf 23 -x264opts ref=4:qpmin=4 -color_primaries 1 -color_trc 1 -colorspace 1 -profile:v main -level 4.0 -f mp4 -ac 2 -c:a aac -b:a 192k -strict -2 -threads 3 -y 'export.mp4'

Sample File: https://goo.gl/yuV0st

Last edited 19 months ago by enevills (previous) (diff)

comment:20 in reply to: ↑ 19 ; follow-up: Changed 19 months ago by cehoyos

  • Cc enevills added

Replying to enevills:

I see the same thing when making x264 files with AAC audio.

Do I understand correctly that this is only reproducible with x264 and not with -vcodec mpeg4 -qscale 2?

comment:21 in reply to: ↑ 20 ; follow-up: Changed 19 months ago by enevills

I just gave -vcodec mpeg4 -qscale 2 and it is 1 frame early. Here is a screenshot from After Effects CC 2015. http://imgur.com/7w40W8M Notice in the bottom right, the playhead is a frame before the waveform beep.

My command was two frames off. Is the issue with the AAC encoding?

Same results seem to happen with v3.0.2 and v2.8.6

Looks like tomorrow I'll be making a 2nd command to offset the video.

Replying to cehoyos:

Replying to enevills:

I see the same thing when making x264 files with AAC audio.

Do I understand correctly that this is only reproducible with x264 and not with -vcodec mpeg4 -qscale 2?

comment:22 in reply to: ↑ 21 ; follow-up: Changed 19 months ago by cehoyos

Replying to enevills:

I just gave -vcodec mpeg4 -qscale 2 and it is 1 frame early.

As said, I don't really believe anything can be "one frame early" but much more important: Is the issue cumulative, does it get worse if you repeat the encoding?

comment:23 in reply to: ↑ 22 Changed 19 months ago by enevills

Replying to cehoyos:

Replying to enevills:

I just gave -vcodec mpeg4 -qscale 2 and it is 1 frame early.

As said, I don't really believe anything can be "one frame early" but much more important: Is the issue cumulative, does it get worse if you repeat the encoding?

It doesn't seem to get worse if I re-encode the output. Even after a few times.

comment:24 Changed 19 months ago by enevills

As a workaround, I'm running this after I transcoder my file. ffmpeg -i x264_file.mov -itsoffset 0.05994 -i x264_file.mov -vcodec copy -acodec copy -map 0:0 -map 1:1 -shortest output.mp4

Note: See TracTickets for help on using tickets.