Opened 6 years ago

Last modified 13 months ago

#6841 open defect

1ms gap in WebM Opus files encoded with ffmpeg

Reported by: Alicia Boya García Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: mkv opus
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

When audio files are encoded with Opus and muxed to WebM in ffmpeg there is a strange 1ms gap before the second audio frame.

Note all Opus frames are 20 ms long. There is a closed set of possible Opus frame durations, i.e. it's not possible to have a 21 ms long Opus frame (see section 2.1.4 of https://tools.ietf.org/html/rfc6716).

How to reproduce:

% curl https://samples.ffmpeg.org/A-codecs/MP3/01%20-%20Charity%20Case.mp3 -o /tmp/any-audio-file.mp3

% ffmpeg -i /tmp/any-audio-file.mp3 -c:a libopus buggy-timestamps.webm

% mkvinfo -v /tmp/buggy-timestamps.webm |grep SimpleBlock |head -n 5
| + SimpleBlock (key, track number 1, 1 frame(s), timestamp 0.000s = 00:00:00.000)
| + SimpleBlock (key, track number 1, 1 frame(s), timestamp 0.021s = 00:00:00.021)
| + SimpleBlock (key, track number 1, 1 frame(s), timestamp 0.041s = 00:00:00.041)
| + SimpleBlock (key, track number 1, 1 frame(s), timestamp 0.061s = 00:00:00.061)
| + SimpleBlock (key, track number 1, 1 frame(s), timestamp 0.081s = 00:00:00.081)

% ffmpeg -version
ffmpeg version 3.3.5 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 7 (GCC)
configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --extra-ldflags='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld ' --extra-cflags='-I/usr/include/nvenc ' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libcdio --enable-indev=jack --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmp3lame --enable-nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-avfilter --enable-avresample --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-libmfx --enable-runtime-cpudetect
libavutil      55. 58.100 / 55. 58.100
libavcodec     57. 89.100 / 57. 89.100
libavformat    57. 71.100 / 57. 71.100
libavdevice    57.  6.100 / 57.  6.100
libavfilter     6. 82.100 /  6. 82.100
libavresample   3.  5.  0 /  3.  5.  0
libswscale      4.  6.100 /  4.  6.100
libswresample   2.  7.100 /  2.  7.100
libpostproc    54.  5.100 / 54.  5.100

Change History (10)

comment:1 by Carl Eugen Hoyos, 6 years ago

Is the issue reproducible with -f lavfi -i sine=d=1?
Please test current FFmpeg git head and provide the command line you tested together with the complete, uncut console output to make this a valid ticket.

in reply to:  1 comment:2 by Alicia Boya García, 6 years ago

Replying to cehoyos:

Is the issue reproducible with -f lavfi -i sine=d=1?

Yes.

Please test current FFmpeg git head and provide the command line you tested together with the complete, uncut console output to make this a valid ticket.

% ~/Apps/ffmpeg-bin/ffmpeg -f lavfi -i sine=d=1 -c:a libopus buggy-timestamps.webm
ffmpeg version N-89088-gce001bb8fc Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7 (GCC)
  configuration: --enable-libopus --prefix=/home/ntrrgc/Apps/ffmpeg-build --bindir=/home/ntrrgc/Apps/ffmpeg-bin
  libavutil      56.  0.100 / 56.  0.100
  libavcodec     58.  3.102 / 58.  3.102
  libavformat    58.  2.100 / 58.  2.100
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  0.101 /  7.  0.101
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
Input #0, lavfi, from 'sine=d=1':
  Duration: N/A, start: 0.000000, bitrate: 705 kb/s
    Stream #0:0: Audio: pcm_s16le, 44100 Hz, mono, s16, 705 kb/s
File 'buggy-timestamps.webm' already exists. Overwrite ? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (pcm_s16le (native) -> opus (libopus))
Press [q] to stop, [?] for help
[libopus @ 0x2a3b480] No bit rate set. Defaulting to 64000 bps.
Output #0, webm, to 'buggy-timestamps.webm':
  Metadata:
    encoder         : Lavf58.2.100
    Stream #0:0: Audio: opus (libopus), 48000 Hz, mono, s16, 64 kb/s
    Metadata:
      encoder         : Lavc58.3.102 libopus
[Parsed_sine_0 @ 0x2a24c20] EOF timestamp not reliable
size=      10kB time=00:00:01.01 bitrate=  80.1kbits/s speed=42.4x    
video:0kB audio:9kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 11.461192%

% mkvinfo -v /tmp/buggy-timestamps.webm | head -n 50
+ EBML head
|+ EBML version: 1
|+ EBML read version: 1
|+ EBML maximum ID length: 4
|+ EBML maximum size length: 8
|+ Doc type: webm
|+ Doc type version: 4
|+ Doc type read version: 2
+ Segment, size 2312240
|+ Seek head (subentries will be skipped)
|+ EbmlVoid (size: 185)
|+ Segment information
| + Timestamp scale: 1000000
| + Multiplexing application: Lavf57.71.100
| + Writing application: Lavf57.71.100
| + Duration: 192.434s (00:03:12.434)
|+ Segment tracks
| + A track
|  + Track number: 1 (track ID for mkvmerge & mkvextract: 0)
|  + Track UID: 1
|  + Lacing flag: 0
|  + Language: und
|  + Codec ID: A_OPUS
|  + Codec delay: 6.500ms (6500000ns)
|  + Seek pre-roll: 80.000ms (80000000ns)
|  + Track type: audio
|  + Audio track
|   + Channels: 2
|   + Sampling frequency: 48000
|   + Bit depth: 16
|  + CodecPrivate, length 19
|+ Cluster
| + Cluster timestamp: 0.000s
| + SimpleBlock (key, track number 1, 1 frame(s), timestamp 0.000s = 00:00:00.000)
|  + Frame with size 389
| + SimpleBlock (key, track number 1, 1 frame(s), timestamp 0.021s = 00:00:00.021)
|  + Frame with size 212
| + SimpleBlock (key, track number 1, 1 frame(s), timestamp 0.041s = 00:00:00.041)
|  + Frame with size 180
| + SimpleBlock (key, track number 1, 1 frame(s), timestamp 0.061s = 00:00:00.061)
|  + Frame with size 58
| + SimpleBlock (key, track number 1, 1 frame(s), timestamp 0.081s = 00:00:00.081)
|  + Frame with size 64
| + SimpleBlock (key, track number 1, 1 frame(s), timestamp 0.101s = 00:00:00.101)
|  + Frame with size 71
| + SimpleBlock (key, track number 1, 1 frame(s), timestamp 0.121s = 00:00:00.121)
|  + Frame with size 65
| + SimpleBlock (key, track number 1, 1 frame(s), timestamp 0.141s = 00:00:00.141)
|  + Frame with size 64
| + SimpleBlock (key, track number 1, 1 frame(s), timestamp 0.161s = 00:00:00.161)

% ~/Apps/ffmpeg-bin/ffmpeg -version
ffmpeg version N-89088-gce001bb8fc Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 7 (GCC)
configuration: --enable-libopus --prefix=/home/ntrrgc/Apps/ffmpeg-build --bindir=/home/ntrrgc/Apps/ffmpeg-bin
libavutil      56.  0.100 / 56.  0.100
libavcodec     58.  3.102 / 58.  3.102
libavformat    58.  2.100 / 58.  2.100
libavdevice    58.  0.100 / 58.  0.100
libavfilter     7.  0.101 /  7.  0.101
libswscale      5.  0.101 /  5.  0.101
libswresample   3.  0.101 /  3.  0.101

comment:3 by Carl Eugen Hoyos, 6 years ago

Component: undeterminedavformat
Keywords: mkv opus added
Reproduced by developer: set
Status: newopen
Version: unspecifiedgit-master

Do I understand correctly that the following command also allows to reproduce?

$ ffmpeg -f lavfi -i sine=d=1 -acodec opus -strict -2 -ac 2 out.webm

in reply to:  3 comment:4 by Alicia Boya García, 6 years ago

Replying to cehoyos:

Do I understand correctly that the following command also allows to reproduce?

$ ffmpeg -f lavfi -i sine=d=1 -acodec opus -strict -2 -ac 2 out.webm

Yes, that command also reproduces the bug.

comment:5 by mkver, 6 years ago

This seems to be a problem with encoder delay (that opus signals explicitly) and rounding timestamps too early. In more detail: The reference opus encoder has a default encoding delay of 312 samples (6.5ms at a sample rate of 48000.) So the first audio block has a PTS of -6.5ms; the second of 13.5ms. Upon conversion to ms precision (the Matroska muxer uses 1/1000 as timebase) the first becomes -7ms, the second 14ms. This is then shifted by default to avoid negative timestamps (depends upon -avoid_negative_ts). Here is the log:

ffmpeg started on 2017-12-01 at 00:48:14
Report written to "ffmpeg-20171201-004814.log"
Command line:
ffmpeg -debug_ts -report -i sin.opus -c copy sin.mka
ffmpeg version N-89341-gd13b8f68d7 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (Rev1, Built by MSYS2 project)
  configuration: --disable-static --enable-shared --enable-avisynth --enable-gcrypt --enable-libopus --enable-libvorbis --enable-gnutls --enable-libbluray --enable-libmfx --enable-libwavpack --enable-gpl --disable-decoder=h264_cuvid --disable-decoder=hevc_cuvid --disable-decoder=mjpeg_cuvid --disable-decoder=mpeg1_cuvid --disable-decoder=mpeg2_cuvid --disable-decoder=mpeg4_cuvid --disable-decoder=vc1_cuvid --disable-decoder=vp8_cuvid --disable-decoder=vp9_cuvid --disable-encoder=dca --disable-encoder=nellymoser --disable-encoder=real_144 --disable-encoder=truehd --disable-encoder=vorbis --disable-encoder=sonic --disable-encoder=sonicls --disable-encoder=amv --disable-encoder=asv1 --disable-encoder=asv2 --disable-encoder=flashsv --disable-encoder=flashsv2 --disable-encoder=roqvideo --disable-encoder=svq1 --disable-encoder=zmbv --disable-encoder=zlib --disable-encoder=snow --disable-encoder=cinepak --disable-encoder=h264_nvenc --disable-encoder=hevc_nvenc --disable-encoder=nvenc_hevc --disable-encoder=nvenc --  libavutil      56.  4.100 / 56.  4.100
  libavcodec     58.  6.102 / 58.  6.102
  libavformat    58.  2.103 / 58.  2.103
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  5.100 /  7.  5.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Splitting the commandline.
Reading option '-debug_ts' ... matched as option 'debug_ts' (print timestamp debugging info) with argument '1'.
Reading option '-report' ... matched as option 'report' (generate a report) with argument '1'.
Reading option '-i' ... matched as input url with argument 'sin.opus'.
Reading option '-c' ... matched as option 'c' (codec name) with argument 'copy'.
Reading option 'sin.mka' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option debug_ts (print timestamp debugging info) with argument 1.
Applying option report (generate a report) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url sin.opus.
Successfully parsed a group of options.
Opening an input file: sin.opus.
[NULL @ 00000000005f5a40] Opening 'sin.opus' for reading
[file @ 00000000005dea00] Setting default whitelist 'file,crypto'
[ogg @ 00000000005f5a40] Format ogg probed with size=2048 and score=100
[ogg @ 00000000005f5a40] 684 bytes of comment header remain
[ogg @ 00000000005f5a40] Before avformat_find_stream_info() pos: 2007 bytes read:2007 seeks:0 nb_streams:1
[ogg @ 00000000005f5a40] All info found
[ogg @ 00000000005f5a40] After avformat_find_stream_info() pos: 2007 bytes read:2007 seeks:0 frames:1
Input #0, ogg, from 'sin.opus':
  Duration: 00:00:00.10, start: 0.000000, bitrate: 160 kb/s
    Stream #0:0, 1, 1/48000: Audio: opus, 48000 Hz, mono, fltp
    Metadata:
      ENCODER         : opusenc from opus-tools 0.1.10-9-gbd65450
Successfully opened the file.
Parsing a group of options: output url sin.mka.
Applying option c (codec name) with argument copy.
Successfully parsed a group of options.
Opening an output file: sin.mka.
[file @ 00000000005debe0] Setting default whitelist 'file,crypto'
Successfully opened the file.
Output #0, matroska, to 'sin.mka':
  Metadata:
    encoder         : Lavf58.2.103
    Stream #0:0, 0, 1/1000: Audio: opus ([255][255][255][255] / 0xFFFFFFFF), 48000 Hz, mono, fltp
    Metadata:
      ENCODER         : opusenc from opus-tools 0.1.10-9-gbd65450
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
demuxer -> ist_index:0 type:audio next_dts:NOPTS next_dts_time:NOPTS next_pts:NOPTS next_pts_time:NOPTS pkt_pts:-312 pkt_pts_time:-0.0065 pkt_dts:-312 pkt_dts_time:-0.0065 off:0 off_time:0
demuxer+ffmpeg -> ist_index:0 type:audio pkt_pts:-312 pkt_pts_time:-0.0065 pkt_dts:-312 pkt_dts_time:-0.0065 off:0 off_time:0
muxer <- type:audio pkt_pts:-7 pkt_pts_time:-0.007 pkt_dts:-7 pkt_dts_time:-0.007 size:300
[matroska @ 00000000005f76c0] get_metadata_duration returned: 0
demuxer -> ist_index:0 type:audio next_dts:-6500 next_dts_time:-0.0065 next_pts:-6500 next_pts_time:-0.0065 pkt_pts:648 pkt_pts_time:0.0135 pkt_dts:648 pkt_dts_time:0.0135 off:0 off_time:0
demuxer+ffmpeg -> ist_index:0 type:audio pkt_pts:648 pkt_pts_time:0.0135 pkt_dts:648 pkt_dts_time:0.0135 off:0 off_time:0
muxer <- type:audio pkt_pts:14 pkt_pts_time:0.014 pkt_dts:14 pkt_dts_time:0.014 size:165
[matroska @ 00000000005f76c0] Writing block at offset 9, size 300, pts 0, dts 0, duration 20, keyframe 1
demuxer -> ist_index:0 type:audio next_dts:13500 next_dts_time:0.0135 next_pts:13500 next_pts_time:0.0135 pkt_pts:1608 pkt_pts_time:0.0335 pkt_dts:1608 pkt_dts_time:0.0335 off:0 off_time:0
demuxer+ffmpeg -> ist_index:0 type:audio pkt_pts:1608 pkt_pts_time:0.0335 pkt_dts:1608 pkt_dts_time:0.0335 off:0 off_time:0
muxer <- type:audio pkt_pts:34 pkt_pts_time:0.034 pkt_dts:34 pkt_dts_time:0.034 size:174
[matroska @ 00000000005f76c0] Writing block at offset 316, size 165, pts 21, dts 21, duration 20, keyframe 1
demuxer -> ist_index:0 type:audio next_dts:33500 next_dts_time:0.0335 next_pts:33500 next_pts_time:0.0335 pkt_pts:2568 pkt_pts_time:0.0535 pkt_dts:2568 pkt_dts_time:0.0535 off:0 off_time:0
demuxer+ffmpeg -> ist_index:0 type:audio pkt_pts:2568 pkt_pts_time:0.0535 pkt_dts:2568 pkt_dts_time:0.0535 off:0 off_time:0
muxer <- type:audio pkt_pts:54 pkt_pts_time:0.054 pkt_dts:54 pkt_dts_time:0.054 size:173
[matroska @ 00000000005f76c0] Writing block at offset 488, size 174, pts 41, dts 41, duration 20, keyframe 1
demuxer -> ist_index:0 type:audio next_dts:53500 next_dts_time:0.0535 next_pts:53500 next_pts_time:0.0535 pkt_pts:3528 pkt_pts_time:0.0735 pkt_dts:3528 pkt_dts_time:0.0735 off:0 off_time:0
demuxer+ffmpeg -> ist_index:0 type:audio pkt_pts:3528 pkt_pts_time:0.0735 pkt_dts:3528 pkt_dts_time:0.0735 off:0 off_time:0
muxer <- type:audio pkt_pts:74 pkt_pts_time:0.074 pkt_dts:74 pkt_dts_time:0.074 size:320
[matroska @ 00000000005f76c0] Writing block at offset 669, size 173, pts 61, dts 61, duration 20, keyframe 1
No more output streams to write to, finishing.
[matroska @ 00000000005f76c0] Writing block at offset 849, size 320, pts 81, dts 81, duration 20, keyframe 1
[matroska @ 00000000005f76c0] end duration = 101
[matroska @ 00000000005f76c0] stream 0 end duration = 101
size=       2kB time=00:00:00.07 bitrate= 206.1kbits/s speed=10.3x    
video:0kB audio:1kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 68.374557%
Input file #0 (sin.opus):
  Input stream #0:0 (audio): 5 packets read (1132 bytes); 
  Total: 5 packets (1132 bytes) demuxed
Output file #0 (sin.mka):
  Output stream #0:0 (audio): 5 packets muxed (1132 bytes); 
  Total: 5 packets (1132 bytes) muxed
0 frames successfully decoded, 0 decoding errors
[AVIOContext @ 00000000023b00c0] Statistics: 14 seeks, 10 writeouts
[AVIOContext @ 00000000005ff2a0] Statistics: 2007 bytes read, 0 seeks

comment:6 by Balling, 2 years ago

But then it is impossible to fix in mkv without increasing precision.

comment:8 by Balling, 13 months ago

The audio is the same even if some changes happen in mkv.

comment:9 by haywirez, 13 months ago

The change only landed today in 6.0: https://git.ffmpeg.org/gitweb/ffmpeg.git/shortlog/n6.0

comment:10 by Balling, 13 months ago

Latest stable is HEAD of master. All those 6.0 branches are not really supposed to be used by anyone. What commit are you talking about?

Note: See TracTickets for help on using tickets.