Incorrect DTS sync in OGG codec

[Tzvi R.]

Hi,

I've noticed incorrect DTS values in OGG encoded audio streams (as part of an OGV file). This happened with a ffmpeg-0.8.7

These are the steps I have taken to reproduce it.

1. Convert a valid and working MP4 (h264/aac) as confirmed by ffprobe to an OGV file using:

   ffmpeg -y -i vid.mp4 -vcodec libtheora -b 1000k -acodec libvorbis -ab 192k vid.ogv
   

2. Probing the "vid.ogv" file using ffprobe shows that during audio frames, the "pos" doesn't change - even for the video frames it seems to repeat ~10 times. See simplified format dump below.

3. While "vid.ogv" plays ok in VLC, after doing a zero transformation on it, it breaks completely and audio goes totally out of sync (video plays ok).

   ffmpeg -y -i vid.ogv -vcodec copy -acodec copy new_vid.ogv
   

4. The new_vid.ogv file no longer plays correctly. Putting some debug prints in ffmpeg.c:output_packet(...) I see the following.
   

A sequence of consecutive audio frames gets the correct DTS values. From 0 to 10941 in steps of ~64.
Then there is a sequence of consecutive video frames. For video the DTS always appears to be correct.
The next sequence of consecutive audio frames now starts at 154048 - instead of being at ~11005

Also notice the negative duration value.

Input #0, ogg, from 'new_vid.ogv':
  Duration: -513359:-56:-35.41, start: 0.000000, bitrate: N/A
    Stream #0.0: Video: theora, yuv420p, 960x540 [PAR 1:1 DAR 16:9], 29.97 tbr, 29.97 tbn, 29.97 tbc
    Stream #0.1: Audio: vorbis, 48000 Hz, stereo, s16, 192 kb/s

If you need any additional information, please let me know.

Thanks,

TR

type=video index=0 dts=0 dts_time=0.000000  [100/2997] duration=1 pos=7414
type=video index=0 dts=1 dts_time=0.033367  [100/2997] duration=1 pos=7414
type=video index=0 dts=2 dts_time=0.066733  [100/2997] duration=1 pos=7414
type=video index=0 dts=3 dts_time=0.100100  [100/2997] duration=1 pos=7414
...
type=video index=0 dts=83 dts_time=2.769436  [100/2997] duration=1 pos=260411
type=audio index=1 dts=0 dts_time=0.000000  [1/48000] duration=0 pos=323479
type=audio index=1 dts=N/A dts_time=N/A [1/48000] duration=0 pos=323479
type=audio index=1 dts=N/A dts_time=N/A [1/48000] duration=0 pos=323479
...
type=audio index=1 dts=N/A dts_time=N/A [1/48000] duration=0 pos=323479
type=audio index=1 dts=N/A dts_time=N/A [1/48000] duration=0 pos=323479
type=video index=0 dts=84 dts_time=2.802803  [100/2997] duration=1 pos=260411
type=video index=0 dts=85 dts_time=2.836170  [100/2997] duration=1 pos=355961
type=video index=0 dts=86 dts_time=2.869536  [100/2997] duration=1 pos=355961
type=video index=0 dts=87 dts_time=2.902903  [100/2997] duration=1 pos=355961
type=video index=0 dts=88 dts_time=2.936270  [100/2997] duration=1 pos=355961
type=video index=0 dts=89 dts_time=2.969636  [100/2997] duration=1 pos=355961
type=video index=0 dts=90 dts_time=3.003003  [100/2997] duration=1 pos=355961
type=video index=0 dts=91 dts_time=3.036370  [100/2997] duration=1 pos=355961
type=video index=0 dts=92 dts_time=3.069736  [100/2997] duration=1 pos=355961
type=video index=0 dts=93 dts_time=3.103103  [100/2997] duration=1 pos=355961
type=video index=0 dts=94 dts_time=3.136470  [100/2997] duration=1 pos=355961
type=video index=0 dts=95 dts_time=3.169837  [100/2997] duration=1 pos=355961
type=video index=0 dts=96 dts_time=3.203203  [100/2997] duration=1 pos=355961
type=video index=0 dts=97 dts_time=3.236570  [100/2997] duration=1 pos=355961
type=video index=0 dts=98 dts_time=3.269937  [100/2997] duration=1 pos=419330
type=video index=0 dts=99 dts_time=3.303303  [100/2997] duration=1 pos=419330
type=video index=0 dts=100 dts_time=3.336670  [100/2997] duration=1 pos=419330
type=video index=0 dts=101 dts_time=3.370037  [100/2997] duration=1 pos=419330
type=video index=0 dts=102 dts_time=3.403403  [100/2997] duration=1 pos=419330
type=video index=0 dts=103 dts_time=3.436770  [100/2997] duration=1 pos=419330
type=video index=0 dts=104 dts_time=3.470137  [100/2997] duration=1 pos=419330
type=video index=0 dts=105 dts_time=3.503504  [100/2997] duration=1 pos=419330
...
type=video index=0 dts=148 dts_time=4.938272  [100/2997] duration=1 pos=545023
type=audio index=1 dts=154048 dts_time=3.209333  [1/48000] duration=0 pos=608173
type=audio index=1 dts=N/A dts_time=N/A [1/48000] duration=0 pos=608173
type=audio index=1 dts=N/A dts_time=N/A [1/48000] duration=0 pos=608173

[Carl Eugen Hoyos]

Complete, uncut console output missing.
Is this reproducible with a publically available ogg sample? With current git head?
Do you believe this is a regression?

[Tzvi R.]

The final result of the "ffmpeg -i out.ogv -vcodec copy -acodec copy out2.ogv"

[Tzvi R.]

The result of converting MP4 to OGG/VORBIS using "ffmpeg -i in.mp4 -vcodec libtheora -b 1000k -acodec libvorbis out.ogv"

[Tzvi R.]

The source MP4 file

[Tzvi R.]

Hi,

I have attached the files I tried (this has been reproduced repeatedly on various source files from various sources - though I always started with MP4).
I haven't tried the git HEAD yet, I will try it later on.

You can view the attached files. Noticed that the last one "out2.ogv" gets the audio completely out of sync by the end of the clip (I view it using VLC).

Here are the commands I used to generate them:

$ ffmpeg -i in.mp4 -vcodec libtheora -b 1000k -acodec libvorbis out.ogv

ffmpeg version 0.8.7, Copyright (c) 2000-2011 the FFmpeg developers

built on Dec 10 2011 09:05:52 with gcc 4.6.1
configuration: --prefix=/opt/ffmpeg-0.8.7 --cpu=core2 --enable-zlib --enable-libxvid --enable-libx264 --enable-libvpx --enable-libvorbis --enable-libspeex --enable-librtmp --enable-libopenjpeg --enable-libmp3lame --enable-libfreetype --enable-libfaac --enable-libdirac --enable-libtheora --enable-bzlib --enable-nonfree --enable-gpl --enable-version3 --enable-shared
libavutil 51. 9. 1 / 51. 9. 1
libavcodec 53. 8. 0 / 53. 8. 0
libavformat 53. 5. 0 / 53. 5. 0
libavdevice 53. 1. 1 / 53. 1. 1
libavfilter 2. 23. 0 / 2. 23. 0
libswscale 2. 0. 0 / 2. 0. 0
libpostproc 51. 2. 0 / 51. 2. 0

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'in.mp4':

Metadata:

major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
creation_time : 1969-12-31 22:00:00
encoder : Lavf53.18.0

Duration: 00:00:14.25, start: 0.000000, bitrate: 876 kb/s

Stream #0.0(und): Video: h264 (High), yuv420p, 720x592 [PAR 257:240 DAR 771:592], 771 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc
Metadata:

creation_time : 1969-12-31 22:00:00

Stream #0.1(und): Audio: aac, 48000 Hz, stereo, s16, 107 kb/s
Metadata:

creation_time : 1969-12-31 22:00:00

[buffer @ 0x88b3c20] w:720 h:592 pixfmt:yuv420p tb:1/1000000 sar:257/240 sws_param:
Output #0, ogg, to 'out.ogv':

Metadata:

major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
creation_time : 1969-12-31 22:00:00
encoder : Lavf53.5.0
Stream #0.0(und): Video: libtheora, yuv420p, 720x592 [PAR 257:240 DAR 771:592], q=2-31, 1000 kb/s, 25 tbn, 25 tbc
Metadata:

creation_time : 1969-12-31 22:00:00

Stream #0.1(und): Audio: libvorbis, 48000 Hz, stereo, s16, 64 kb/s
Metadata:

creation_time : 1969-12-31 22:00:00

Stream mapping:

Stream #0.0 -> #0.0
Stream #0.1 -> #0.1

$ ffmpeg -i out.ogv -vcodec copy -acodec copy out2.ogv

Thanks,

TZvi.

[Tzvi R.]

The result of running "ffmpeg -y -i out.ogv -vcodec libvpx -acodec copy out2.webm"

[Tzvi R.]

Hi,

I have tested against the git HEAD (commit 8349dbfe46140373082f43ca090dac22696afbb4 from Mon Dec 12 14:48:53 2011 +0200) and the problem persists.

You can easily reproduce it using these two commands (I have attached the relevant files).

• ffmpeg -y -i in.mp4 -vcodec libtheora -b 1000k -acodec libvorbis out.ogv
• ffmpeg -y -i out.ogv -vcodec copy -acodec copy out2.ogv
• The resulting out2.ogv file is broken (the audio drifts quickly and goes out of sync with the video). The first out.ogv file appears to be fine.

This appears to be a breakage in the OGG reader, since it happens whenever you COPY or possibly even read OGG streams.
It's much worse actually when COPYing the stream into a webm container, see attachment out2.webm, where the audio is choppy and broken.

• ffmpeg -y -i out.ogv -vcodec libvpx -acodec copy out2.webm
• The resulting out2.webm has choppy and illegible audio.

See attached files for examples (you can test the above commands against).

I've traced ffmpeg calls to av_read_frame() and I think I see a problem.

1. It reads a stream of audio packets starting with DTS=0 (the following packets have no DTS but ffmpeg interpolates it from the first valid one it received)
2. It then reads a stream of video packets.
3. Another stream audio packets comes in. Again, only the first one has a DTS defined for it, however, that values is not continuous with the last interpolated value in (1). While the last packet in (1) ends with DTS=10941, the first packet in (3) starts with DTS=154048. There is an unexplained gap here since the DTS has been advancing in steps of 64.

I don't know if that's normal however this doesn't happen in other containers (like MP4 & WEBM for example).

I hope I have provided enough information to go by (output of the calls appears in my previous response). If not, please let me know what else should I supply.

Thanks,

Tzvi

[Carl Eugen Hoyos]

Replying to zvi:

• ffmpeg -y -i in.mp4 -vcodec libtheora -b 1000k -acodec libvorbis out.ogv
• ffmpeg -y -i out.ogv -vcodec copy -acodec copy out2.ogv
• The resulting out.ogv file is broken (the audio drifts quickly and goes out of sync with the video).

out.ogv plays fine afaict.
(You should be able to edit your message.)

The problem is for example reproducible with
ffmpeg -i out.ogv -acodec copy out.mkv
Resulting file is heavily out-of-sync.

[reimar]

Should be a duplicate of issue #13.

[llogan]

Closing as duplicate of issue #13 as indicated by reimar.

[Carl Eugen Hoyos]

Fixed by Justin together with ticket #13.