memory leak using h264_qsv to encode under certain resolutions

[carlchen]

There are memory leaks when I use h265_qsv to encoder a file under certain resolutions, I set the target file's resolution to 1920x1080 or 1280x720, every thing is ok, but if I set the resolution to 720x480 or 320x288,the memory will increase continuously。I find some one mentioned about it
​http://trac.ffmpeg.org/ticket/8021
It cloed but not fix.

[Carl Eugen Hoyos]

Please provide the command line that you tested together with the complete, uncut console output and test with valgrind or a similar tool.
Note that increasing memory consumption is not the same as a memory leak.

[Linjie.Fu]

Please provide the command line that you tested together with the complete, uncut console output and test with valgrind or a similar tool.
Note that increasing memory consumption is not the same as a memory leak.

Please also provide platform/OS information and input clips for reproducing, thanks.

[lveilleux]

This leak is very real and reproducible when using the APIs programmatically, although it may be difficult to detect using the one-pass cli.

Using version 4.2.3 LGPL build from Zeranoe on Windows 64-bit. Below is a snippet of code. If you wrap this into a function and call it a few times, you will find there are references to the frame that are not released by the codec and memory usage keeps increasing. This does not happen with other codecs I tried (h264_nvenc, h264_amf and mpeg4), and it does not happen with h264_qsv when resolution is 1920x1080 or 1280x720.

Find encoder and allocate context
AVCodecContext *m_c = avcodec_alloc_context3(avcodec_find_encoder_by_name("h264_qsv"));

Setup basic parameters
m_c->width = 1280;
m_c->height = 1024;
m_c->gop_size = 200;
m_c->bit_rate = 1000000;
m_c->time_base.den = 90000;
m_c->time_base.num = 3000;
m_c->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;

m_c->pix_fmt = AV_PIX_FMT_NV12;
av_opt_set(m_c->priv_data, "preset", "faster", 0);
av_opt_set_int(m_c->priv_data, "forced_idr", 1, 0);
av_opt_set_int(m_c->priv_data, "look_ahead", 1, 0);
m_c->global_quality = 30;

In case of failure to open, close codec and free memory
if (avcodec_open2(m_c, m_c->codec, NULL) < 0)
{

avcodec_close(m_c);
av_free(m_c);
m_c = NULL;
return;

}

AVFrame *m_frame = av_frame_alloc();

Initialize the frame basic parameters and allocate image buffers
m_frame->format = m_c->pix_fmt;
m_frame->width = m_c->width;
m_frame->height = m_c->height;

AVBufferRef* m_buffer = av_buffer_alloc(av_image_get_buffer_size(m_c->pix_fmt, m_c->width, m_c->height, 1) + AV_INPUT_BUFFER_PADDING_SIZE);
m_frame->buf[0] = m_buffer;

av_image_fill_linesizes(m_frame->linesize, (AVPixelFormat)m_frame->format, m_frame->width);
av_image_fill_pointers(m_frame->data, (AVPixelFormat)m_frame->format, m_frame->height, m_buffer->data, m_frame->linesize);

for (int i=0; i<1000; i++)
{

m_frame->pts = i * m_c->time_base.num;
avcodec_send_frame(m_c, m_frame);

AVPacket pkt = { NULL };
avcodec_receive_packet(m_c, pkt);

}

avcodec_close(m_c);
av_free(m_c);

m_frame->buf[0] = NULL; freed as m_buffer below
av_frame_unref(m_frame);
av_frame_free(&m_frame);
av_buffer_unref(&m_buffer);

I ran this on i7-8650U with up to date driver from Dell.

[lveilleux]

Replying to cehoyos:
Which info is missing?

[Carl Eugen Hoyos]

Since there is no release support on this bug tracker, the first missing information is confirmation that you tested with current FFmpeg.

Apart from that, please see my message above: Post output from valgrind or a similar tool if you want to report a leak (which is not the same as excessive memory usage) in addition to the command line you tested together with the complete, uncut console output if the issue is reproducible with ffmpeg, the alternative is to post actual source code that can be compiled (what you posted cannot be compiled with any C compiler).

[lveilleux]

Source code + exec to reproduce + test results

[lveilleux]

I added an attachment, containing VS2019 solution/project/source. I linked this with nightly build ffmpeg-20200615-9d80f3e-win64-shared (GPL).

I ran memory check using sysinternals' VMMap, which is also included in the attachment (both the utility and the results). Basically, the small Win64 console app runs 3 tests, there's a "pause" command in between each to allow taking a snapshot of memory. The tests are:

• Open h264_qsv, encode 60 frames at 1280x720, close codec, run that 5 times (heap size increases just a few KBs after this)
• Open h264_qsv, encode 60 frames at 1280x1024, close codec, run that 5 times (heap size increases 80MB after this)
• Open h264_qsv, encode 60 frames at 1920x1080, close codec, run that 5 times (heap size increases just a few KBs after this)

[Linjie.Fu]

Did some quick tests on linux, however didn't reproduce the memory leak with following cmdline:

valgrind --tool=memcheck --leak-check=full --log-file=webm_log.txt /root/build/ffmpeg/ffmpeg_g -hwaccel qsv -init_hw_device qsv=hw -filter_hw_device hw -v verbose -f rawvideo -video_size 1920x1080 -pix_fmt yuv420p -i /root/fulinjie/1080p_blue_sky.yuv -an -vf scale=w=1280:h=1024,hwupload=extra_hw_frames=64,format=qsv -c:v h264_qsv -vframes 100 -y ./qsv.h264

Hence,
Is it windows specific issue?
And did you try to encode with FFmpeg cmdline?

[lveilleux]

* Correction: the leak is not real :)
Using the proper av_frame_get_buffer to initialize the AVFrame, ref counts are properly handled and qsvenc releases correctly

I was using some weirdo code that worked for many other cases, but not in this one. I also ended up padding/aligning height of the AVFrame to make sure qsvenc was not making unnecessary copies of it to increase performance. Thanks for your help.