Allow to drop frames to keep A/V sync

[Carl Eugen Hoyos]

A user uploaded a HD recording of a Lumia cellphone which shows heavy A/v desync issues with QT because audio has "holes" - especially noticeable after 77 seconds. FFplay, MPlayer and WMP speed up video to keep A/V sync.
FFmpeg by default looses A/V sync, encoding with -async 1 works fine but produces silent moments.
There should be an option to drop video frames to keep A/V sync if another stream is missing content, just like the media players skip these frames.

$ ffmpeg -i source2.mp4 -qscale 2 out.avi
ffmpeg version N-73169-gf5822ea Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      54. 27.100 / 54. 27.100
  libavcodec     56. 45.100 / 56. 45.100
  libavformat    56. 38.102 / 56. 38.102
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 18.100 /  5. 18.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.100 /  1.  2.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'source2.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp423gp6isomM4V
    creation_time   : 2015-06-20 18:05:00
    encoder         : 01
    encoder-eng     : 01
  Duration: 00:01:34.38, start: 0.000000, bitrate: 17730 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, unknown/smpte170m/unknown), 1920x1080 [SAR 1:1 DAR 16:9], 17601 kb/s, 30.01 fps, 30 tbr, 30k tbn, 60 tbc (default)
    Metadata:
      creation_time   : 2015-06-20 18:05:00
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 153 kb/s (default)
    Metadata:
      creation_time   : 2015-06-20 18:05:00
      handler_name    : AudioHandler
Please use -q:a or -q:v, -qscale is ambiguous
Output #0, avi, to 'out.avi':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp423gp6isomM4V
    ISFT            : Lavf56.38.102
    Stream #0:0(und): Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn, 30 tbc (default)
    Metadata:
      creation_time   : 2015-06-20 18:05:00
      handler_name    : VideoHandler
      encoder         : Lavc56.45.100 mpeg4
    Stream #0:1(und): Audio: ac3 ([0] [0][0] / 0x2000), 48000 Hz, mono, fltp, 96 kb/s (default)
    Metadata:
      creation_time   : 2015-06-20 18:05:00
      handler_name    : AudioHandler
      encoder         : Lavc56.45.100 ac3
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mpeg4 (native))
  Stream #0:1 -> #0:1 (aac (native) -> ac3 (native))
Press [q] to stop, [?] for help
frame= 2826 fps=130 q=2.0 Lsize=  560532kB time=00:01:34.36 bitrate=48659.3kbits/s
video:559304kB audio:1082kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.026151%

[Carl Eugen Hoyos]

​http://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket4674/

[JouMxyzptlk]

Confirmed!
-async 1 gives me the same results as "playing with player and capture it", aka keep the sync at the cost of audio gaps (which is fine for now, aka best result up to now). Ffmpeg method is much better than play-record (especially since I can batch it).
A "drop or insert frames without mercy" variant which keeps the audio gapless would be indeed very nice and preferred by me.
If the "source2.mp4" is needed again for testing just say, it is currently removed from the webspace.

[Carl Eugen Hoyos]

Replying to JouMxyzptlk:

-async 1 gives me the same results as "playing with player and capture it"

No, clearly not: All players speed up video playback to reach gapless audio, this is not possible with ffmpeg.

[Carl Eugen Hoyos]

Another real-world example file is in ​http://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket4720/

[Carl Eugen Hoyos]

An additional sample is In\ the\ Heat\ of\ the\ Night_WHDHDT2_2014_11_23_18_58_00_cut.wtv in ​http://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket4166/