Opened 5 years ago

Closed 5 years ago

#3231 closed defect (fixed)

Muxing libvorbis audio fails; no sound in the output file

Reported by: gyunaev Owned by:
Priority: normal Component: documentation
Version: git-master Keywords: muxing
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: yes

Description

There seem to be an issue related to muxing vorbis audio in ffmpeg, both 2.1.1 release and GIT head. The issue is fairly easy to reproduce with the muxing.c example in the FFmpeg doc/example directory by issuing the following command:

./muxing test.webm

The resulting test.webm will play video fine with ffplay, and will have the audio track, but will output no sound. It does not generate any errors either.

This issue is related to vorbis and not libvpx/webm container, which could be further proved by adding the following two lines to force muxing test.ogg into theora/vorbis:

--- muxing.c.orig       2013-12-18 01:25:24.126903125 -0800
+++ muxing.c    2013-12-18 01:29:12.643882209 -0800
@@ -487,6 +487,7 @@
     video_st = NULL;
     audio_st = NULL;
 
+       fmt->audio_codec = AV_CODEC_ID_VORBIS;
     if (fmt->video_codec != AV_CODEC_ID_NONE) {
         video_st = add_stream(oc, &video_codec, fmt->video_codec);
     }

(this patch is necessary because by default it attempts to mux test.ogg into theora/flac - which by the way fails right away). And running make and executing as following:

./muxing test.ogg

The resulted test.ogg will work exactly as test.webm above - ffplay will play only video and no sound.

Further investigating this issue it looks like vorbis audio requires setting the frame->pts. Applying the following patch to muxing.c fixes the audio output:

--- muxing.c.orig       2013-12-18 01:25:24.126903125 -0800
+++ muxing.c    2013-12-18 01:36:03.566844598 -0800
@@ -258,6 +258,9 @@
         dst_nb_samples = src_nb_samples;
     }
 
+    static int audio_pts = 0;
+    frame->pts = audio_pts;
+    audio_pts += dst_nb_samples;
     frame->nb_samples = dst_nb_samples;
     avcodec_fill_audio_frame(frame, c->channels, c->sample_fmt,
                              dst_samples_data[0], dst_samples_size, 0);

With this patch theora/vorbis audio mixes fine, and ./muxing test.ogg plays the sound. However this patch breaks muxing every other format - for example, .patched /muxing test.avi uses mpeg4/vorbis and loses sound after 4 seconds. It is also seem to be exclusively vorbis-specific. libtheora+flac in ogg muxes without problem, while mpeg4+vorbis in AVI experiences the audio loss. More important, it doesn't fix webm libvpx/libvorbis, so it may be a different issue.

Change History (10)

comment:1 Changed 5 years ago by gyunaev

Also my libvorbis version is 1.3.3 and my system is OpenSuSE Linux 13.1 on x86_64

comment:2 Changed 5 years ago by cehoyos

  • Component changed from avcodec to documentation
  • Keywords vorbis removed
  • Priority changed from normal to minor

comment:3 Changed 5 years ago by gyunaev

Could you please provide more feedback? If this is purely the documentation issue, why the FFmpeg example does not work?

comment:4 Changed 5 years ago by cehoyos

  • Version changed from unspecified to git-master

muxing.c is part of the FFmpeg documentation, but I wonder if it is ok to first change the source code and then complain that it does not work (I didn't test myself). I leave it to the documentation maintainers to decide.

comment:5 follow-up: Changed 5 years ago by gyunaev

Thank you very much for following this up. You are correct, but please keep in mind that actual problem reported is that muxing WEBM (libvpx + libvorbis) does not work at all with unmodified muxing.c. As of right now I'm not sure whether the problem is in muxing.c or in libavcodec/libavformat since after spending two days of it I was never able to actually mux vpx and vorbis in the container which would play audio. No matter what I do it results exactly as what happens if you run unmodified ./muxing test.webm

The modifications below are there to describe why I believe the problem is in vorbis codec, and not in Theora coder or Webm container. They're not to be applied to the released muxing.c, otherwise I'd posted them into -devel.

comment:6 in reply to: ↑ 5 Changed 5 years ago by cehoyos

Replying to gyunaev:

Thank you very much for following this up. You are correct, but please keep in mind that actual problem reported is that muxing WEBM (libvpx + libvorbis) does not work at all with unmodified muxing.c.

So this is not a bug report but a documentation enhancement request?

As of right now I'm not sure whether the problem is in muxing.c or in libavcodec/libavformat since after spending two days of it I was never able to actually mux vpx and vorbis in the container which would play audio.

Does it work with ffmpeg?

comment:7 follow-up: Changed 5 years ago by gyunaev

Yes, either documentation or source code enhancement request. It is basically "please make it working in the default scenario" request.

I have no means to test it with ffmpeg since it operates differently from muxing.c - it uses input file(s) while muxing.c generates the data. So yes, ffmpeg would have the same behavior if it uses the same code as muxing.c.

comment:8 in reply to: ↑ 7 Changed 5 years ago by cehoyos

Replying to gyunaev:

I have no means to test it with ffmpeg since it operates differently from muxing.c - it uses input file(s) while muxing.c generates the data. So yes, ffmpeg would have the same behavior if it uses the same code as muxing.c.

Could you elaborate?
Which code could ffmpeg use that muxing.c cannot use? And why can't ffmpeg not process the same data as muxing.c?

Version 0, edited 5 years ago by cehoyos (next)

comment:9 Changed 5 years ago by gyunaev

As far as I know, FFmpeg cannot generate video/audio as muxing.c does. It works with the external input audio and video streams, and in converting those it uses the DTS/PTS information from those streams.

muxing.c however does not use any external streams, and generates the video and audio streams internally. Thus it cannot rely on provided PTS/DTS and has to supply it. muxing.c does it for video, but not for audio. This works fine with the majority of formats, but it does not work with WEBM or theora/vorbis.

This could mean that FFmpeg coders are ok and muxing.c needs fixing (because changing in as above fixes the output for theora/vorbis, although not for WebM). This could also mean there's a bug in relevant codec/format parser in FFmpeg, and the change above just avoids this bug in some way. This theory could be further supported by the fact that any other format except Vorbis audio muxes fine with the unmodified example, while any format containing Vorbis audio fail to mux properly.

This is important issue because the muxing example is the only one which is relevant for applications such as my Karaoke Lyrics Editor (which is a free GPL app) which generates an AV interleaved file based on the user entered data.

comment:10 Changed 5 years ago by saste

  • Analyzed by developer set
  • Keywords muxing added
  • Priority changed from minor to normal
  • Reproduced by developer set
  • Resolution set to fixed
  • Status changed from new to closed

Fixed in:

commit 094c500caf0c2a67792f2ee8238f065926aebfc4
Author: Stefano Sabatini <stefasab@gmail.com>
Date:   Wed Jan 8 15:43:25 2014 +0100

    examples/muxing: set timestamps in output audio packet
    
    In particular, fix trac ticket #3231.
Note: See TracTickets for help on using tickets.