Opened 6 months ago

Last modified 5 months ago

#10848 new defect

Crackling when concatenating audio introduced in commit d85c6aba

Reported by: jonnyburger Owned by:
Priority: normal Component: undetermined
Version: unspecified Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

We found that when upgrading to FFmpeg 6.1, a regression in audio quality has occurred when concatenating PCM-16 audio chunks into AAC.

After bisecting 2000 commits, we find that the behavior has changed in https://github.com/FFmpeg/FFmpeg/commit/d85c6aba0cf27db2a6c4dfa3452cfb9c248d1b4a. This commit seems to have changed the test fixtures as well, indicating that it makes inadverted changes to the output.

As of writing, the crackling is still on master.

Important detail: The crackling is only audible on QuickTime on macOS with a relatively high volume. Playing the video for example in a browser does not result in audio artifacts.

Repro: https://jonnyburger.s3.eu-central-1.amazonaws.com/crackling.zip
Also uploaded as crackling.zip on the VideoLAN file uploader.

To reproduce, you need to download the PCM chunks and concat them using:

ffmpeg -loglevel verbose -r 30 -f concat -safe 0 -i files.txt -c:v copy -c:a aac -b:a 320k output.mp4

Since the codebase has further evolved and it's not possible to just revert the commit, I derived a hacky fix for anyone else facing this issue (might break something else): https://jonnyburger.s3.eu-central-1.amazonaws.com/fix-aac.patch

Attachments (1)

compared-audio.zip (284.5 KB ) - added by jonnyburger 5 months ago.
Comparing output of FFmpeg 6.0 and 6.1

Download all attachments as: .zip

Change History (10)

comment:1 by jonnyburger, 5 months ago

For those interested, there is an even narrower patch:

diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c
index fcee8b65ac..07edcef4f2 100644
--- a/fftools/ffmpeg_dec.c
+++ b/fftools/ffmpeg_dec.c
@@ -214,7 +214,7 @@ static void audio_ts_process(void *logctx, Decoder *d, AVFrame *frame)
     // on samplerate change, choose a new internal timebase for timestamp
     // generation that can represent timestamps from all the samplerates
     // seen so far
-    tb = audio_samplerate_update(logctx, d, frame);
+    tb = frame->time_base;
     pts_pred = d->last_frame_pts == AV_NOPTS_VALUE ? 0 :
                d->last_frame_pts + d->last_frame_duration_est;


Unfortunately I don't have the expertise to understand what is going on and submit a proper patch to FFmpeg.

Last edited 5 months ago by jonnyburger (previous) (diff)

comment:2 by Marton Balint, 5 months ago

chunk6 is missing from the sample, so can't really reproduce. Also I suggest try reproducing with sine wave as source, much easier to hear any cracks. What is returned from audio_samplerate_update and what is in frame->time_base?

comment:3 by Balling, 5 months ago

when concatenating PCM-16 audio chunks into AAC.

Does it happen if you just get wav pcm? No encode to aac.

Marton... You can just copy paste one of the segments into file called chunk_00000006, seriously?

Last edited 5 months ago by Balling (previous) (diff)

comment:4 by Balling, 5 months ago

Anyway I cannot reproduce that. Looking it up in Audacityy the segments all got concatenated together correctly.

comment:5 by jonnyburger, 5 months ago

Thanks a lot for looking into it!
I am working on providing a better reproduction and answering your questions.
Unfortunately chunk000006 had a trailing whitespace.
I find the audio sounds different in Chrome than it does in QuickTime.
It indicates a problem with the PTS, not with the waveform I suppose.

I’ll follow up on this with a better example.

comment:6 by jonnyburger, 5 months ago

I have now uploaded a second repro!

Filename: reproduction-two.zip
Alternate URL: https://jonnyburger.s3.eu-central-1.amazonaws.com/reproduction-two.zip

This one uses a sine sound.
It also only leads to audio artifacts in QuickTime.

I also compiled a version of FFmpeg n6.1 tag with the following patch applied:

diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c
index fcee8b65ac..3b27e15279 100644
--- a/fftools/ffmpeg_dec.c
+++ b/fftools/ffmpeg_dec.c
@@ -215,6 +215,9 @@ static void audio_ts_process(void *logctx, Decoder *d, AVFrame *frame)
     // generation that can represent timestamps from all the samplerates
     // seen so far
     tb = audio_samplerate_update(logctx, d, frame);
+    av_log(logctx, AV_LOG_WARNING,
+                "Remotion Repro: tb: %d sr: %d\n", tb, frame->sample_rate);
+
     pts_pred = d->last_frame_pts == AV_NOPTS_VALUE ? 0 :
                d->last_frame_pts + d->last_frame_duration_est;

It prints out:

Remotion Repro: tb: 1 sr: 48000

Is it bad to use %d? Both with this and the fix I got the same log.
I can do more logging if you like me to.

I also added my compiled version of FFmpeg in the ZIP.

comment:7 by jonnyburger, 5 months ago

I managed to properly log now:

frame->time_base: 1/1000
audio_samplerate_update(logctx, d, frame) -> 1/48000

comment:8 by Balling, 5 months ago

Can you attach two aac encoded files, after and before the patch?

by jonnyburger, 5 months ago

Attachment: compared-audio.zip added

Comparing output of FFmpeg 6.0 and 6.1

in reply to:  1 comment:9 by jonnyburger, 5 months ago

Absolutely, here is the difference between the outputs. 6_0.mp4 sounds good, 6_1 does not. The change in source code is only:

diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c
index fcee8b65ac..07edcef4f2 100644
--- a/fftools/ffmpeg_dec.c
+++ b/fftools/ffmpeg_dec.c
@@ -214,7 +214,7 @@ static void audio_ts_process(void *logctx, Decoder *d, AVFrame *frame)
     // on samplerate change, choose a new internal timebase for timestamp
     // generation that can represent timestamps from all the samplerates
     // seen so far
-    tb = audio_samplerate_update(logctx, d, frame);
+    tb = frame->time_base;
     pts_pred = d->last_frame_pts == AV_NOPTS_VALUE ? 0 :
                d->last_frame_pts + d->last_frame_duration_est;


Note: See TracTickets for help on using tickets.