Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#3525 closed defect (invalid)

AAC to PCM produced a lot of noise!

Reported by: kaienfr Owned by:
Priority: normal Component: undetermined
Version: git-master Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

In git-master ffmpeg, converting a AAC audio to PCM produced a lot of noise, even we did resampling from AV_SAMPLE_FMT_FLTP to AV_SAMPLE_FMT_S16.

While in old ffmpeg, aac is converted directly into 16-bit PCM and without any noise. Is it a bug or something? I'd like to know how to produce a proper PCM in the new version of ffmpeg? Thanks a lot!

Here is the information about my test AAC audio which reads by new ffmpeg:

Stream #0:0: Audio: aac, 44100 Hz, stereo, fltp, 122 kb/s

(the format fltp is shown as s16 in old ffmpeg, and flt in decoder)

I've tried three ways to resampling the output FLTP to S16LE,

  1. using swr_convert
  2. using avresample_convert
  3. convert manualy

But all of them yield the same result. The sound quality is really too bad, very slow and out of tune, with a lot of noise too.

My resampling code is as follows, please have a look what's missing

void resampling(AVFrame* frame_, AVCodecContext* pCodecCtx, int64_t want_sample_rate, uint8_t* outbuf){
    SwrContext      *swrCtx_ = 0;
    AVAudioResampleContext *avr = 0;

    // Initializing the sample rate convert. We only really use it to convert float output into int.
    int64_t wanted_channel_layout = AV_CH_LAYOUT_STEREO;

#ifdef AV_SAMPLEING
    avr = avresample_alloc_context();
    av_opt_set_int(avr, "in_channel_layout", frame_->channel_layout, 0);
    av_opt_set_int(avr, "out_channel_layout", wanted_channel_layout, 0);
    av_opt_set_int(avr, "in_sample_rate", frame_->sample_rate, 0);
    av_opt_set_int(avr, "out_sample_rate", 44100, 0);
    av_opt_set_int(avr, "in_sample_fmt", pCodecCtx->sample_fmt, 0); //AV_SAMPLE_FMT_FLTP
    av_opt_set_int(avr, "out_sample_fmt", AV_SAMPLE_FMT_S16, 0);
    av_opt_set_int(avr, "internal_sample_fmt", pCodecCtx->sample_fmt, 0);
    avresample_open(avr);
    avresample_convert(avr, &outbuf, frame_->linesize[0], frame_->nb_samples, frame_->extended_data, frame_->linesize[0], frame_->nb_samples);
    avresample_close(avr);
    return;
#endif

#ifdef USER_SAMPLEING
    if (pCodecCtx->sample_fmt == AV_SAMPLE_FMT_FLTP)
    {
            int nb_samples = frame_->nb_samples;
            int channels = frame_->channels;
            int outputBufferLen = nb_samples & channels * 2;
            auto outputBuffer = (int16_t*)outbuf;

            for (int i = 0; i < nb_samples; i++)
            {
                    for (int c = 0; c < channels; c++)
                    {
                            float* extended_data = (float*)frame_->extended_data[c];
                            float sample = extended_data[i];
                            if (sample < -1.0f) sample = -1.0f;
                            else if (sample > 1.0f) sample = 1.0f;
                            outputBuffer[i * channels + c] = (int16_t)round(sample * 32767.0f);
                    }
            }
            return;
    }
#endif
    swrCtx_ = swr_alloc_set_opts(
            NULL, //swrCtx_,
            wanted_channel_layout,
            AV_SAMPLE_FMT_S16,
            want_sample_rate,
            pCodecCtx->channel_layout,
            pCodecCtx->sample_fmt,
            pCodecCtx->sample_rate,
            0,
            NULL);

    if (!swrCtx_ || swr_init(swrCtx_) < 0) {
            printf("swr_init: Failed to initialize the resampling context");
            return;
    }

    // convert audio to AV_SAMPLE_FMT_S16
    int swrRet = swr_convert(swrCtx_, &outbuf, frame_->nb_samples, (const uint8_t **)frame_->extended_data, frame_->nb_samples);
    if (swrRet < 0) {
            printf("swr_convert: Error while converting %d", swrRet);
            return;
    }
}


I've also noticed that in ffplay, the aac audio is playing without noise. So what is the problem it could be? what is the right way of the conversion? is there something wrong in in decode part or resampling part or something else (such as we must use filter...)? Please help!

Thanks a lot in advance!

Change History (6)

comment:1 in reply to: ↑ description Changed 5 years ago by cehoyos

  • Priority changed from critical to normal
  • Resolution set to invalid
  • Status changed from new to closed

Replying to kaienfr:

In git-master ffmpeg, converting a AAC audio to PCM produced a lot of noise, even we did resampling from AV_SAMPLE_FMT_FLTP to AV_SAMPLE_FMT_S16.

You may of course use swresample directly but consider using the aresample filter instead.

While in old ffmpeg, aac is converted directly into 16-bit PCM and without any noise. Is it a bug or something?

No, it is a performance improvement.

swrCtx_ = swr_alloc_set_opts(

NULL, swrCtx_,

Why aren't you calling swr_alloc()?
If you want to use libswresample directly, you can look at libavfilter/af_aresample.c for an example on how to use the library.

I've also noticed that in ffplay, the aac audio is playing without noise.

Did you also test ffmpeg -i file.aac out.wav?
Please understand that this is a bug tracker, not a support forum, see https://ffmpeg.org/contact.html

comment:2 Changed 5 years ago by kaienfr

Thanks a lot for your response. I will try call swr_alloc() for testing, but I am not sure if it will work since in fact, it works quite well in the old version for a long time...

I have asked this question on several forums such as stack overflow http://stackoverflow.com/questions/22822515/ffmpeg-resampling-from-av-sample-fmt-fltp-to-av-sample-fmt-s16-got-very-bad-so

and chinavideo http://bbs.chinavideo.org/forum.php?mod=viewthread&tid=19551#lastpost

but always no response and no correct solution. It's not only me but many people have the same problem when converting from AAC to PCM in new ffmpeg libs on google, and always no solution. That's why I doubt it could be a bug and ask question here.

I will try it and give you feedback soon. Thanks a lot in advance!

Version 1, edited 5 years ago by kaienfr (previous) (next) (diff)

comment:3 Changed 5 years ago by kaienfr

Nothing changed with swr_alloc()

Here is the comparaisons of decoded output with and without resampling produced by the same code with different ffmpeg versions. The figure is made with "cool edit pro"

  1. old ffmpeg without resampling (44100, stereo, s16le)

https://cloud.githubusercontent.com/assets/5306849/2605904/910d68fa-bb49-11e3-9baa-2ccd70b22645.JPG

  1. new ffmpeg without resampling (44100, stereo, 32bit IEEE float 0.24, Impossible to open with s16le since they are all noise, if you want to see, I can also show you that picture...):

https://cloud.githubusercontent.com/assets/5306849/2605902/91075460-bb49-11e3-8be2-426ce9d779d8.JPG

  1. old ffmpeg with resampling (44100, stereo, s16le):

https://cloud.githubusercontent.com/assets/5306849/2605905/910e8050-bb49-11e3-82db-72d3c7eaaf99.JPG

  1. new ffmpeg with resampling (44100, stereo, s16le):

https://cloud.githubusercontent.com/assets/5306849/2605903/91089546-bb49-11e3-9ef4-3646d2f94775.JPG

You can observe quite a different between the outputs. Old ffmpeg is all right for both decode and resampling, well new ffmpeg does not work well on neither of them.

I haven't tried filter, I will do it later, and tell you what happened. Thanks again for your futher helps!
(PS: testing on ffmpeg.exe and ffplay.exe are all right.)

Last edited 5 years ago by kaienfr (previous) (diff)

comment:4 Changed 5 years ago by kaienfr

  • Resolution invalid deleted
  • Status changed from closed to reopened

I've tried your suggestion to use aresample filter, this is the code:

	ReSampleContext *rs_ctx = NULL;
        // resample to 44100, stereo, s16
	rs_ctx = av_audio_resample_init(
		2, pCodecCtx->channels,
		44100, pCodecCtx->sample_rate,
		AV_SAMPLE_FMT_S16, pCodecCtx->sample_fmt,
		16, 10, 0, 1);
        outbuff = (uint8_t*)av_malloc(AVCODEC_MAX_AUDIO_FRAME_SIZE);

       .....................

	// resampling
	after_sampled_len = audio_resample(rs_ctx, (short *)outbuff, (short *)pFrame->extended_data[0], pFrame->nb_samples);

I've gotten the following errors at the function audio_resample:

[audioresample @ 023AC060] Audio sample format conversion failed

The same code run perfactly with old ffmpeg and got wonderful result!

Well, what kind of problem this should be in new ffmpeg? Obviously, there is something abnormal in the new version.
If the way to use it is changed, PLEASE clarify it. I am looking forward to your help. Thank you very much indeed!
Problem not solved, so reopen this ticket.

Last edited 5 years ago by kaienfr (previous) (diff)

comment:5 Changed 5 years ago by kaienfr

  • Resolution set to fixed
  • Status changed from reopened to closed

Well I figure it out. comparing to the old ffmpeg, It seems that in new ffmpeg, each frame is twice larger than old one, so I guess that the output size could be larger estimated in new version. So I tried devided the size of the resampled data by 2, well I got the perfact audio

https://cloud.githubusercontent.com/assets/5306849/2609128/8806eb40-bb72-11e3-9a11-e2b31f98ca97.JPG

This difference seems for aac audio between old and new ffmpeg. Hope it will helpful to people who is still under the trap of the new ffmpeg issues. Thanks all the same cehoyos for your apply to my question.

comment:6 Changed 5 years ago by cehoyos

  • Resolution changed from fixed to invalid
Note: See TracTickets for help on using tickets.