Memory leakage in acceletared encoding (h264_vaapi and hevc_vaapi)

[hsyrja]

Summary of the bug:
How to reproduce:

% ffmpeg -y -vaapi_device /dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format yuv420p -i file.264 -an -vf format=nv12,hwupload -c:v hevc_vaapi -quality 1 -f mpegts /dev/null
ffmpeg version N-81786-g2586668
built on with --enable-nonfree --enable-gpl --enable-version3 --enable-vaapi --enable-libx264
gcc 4.9.2 (Debian 4.9.2-10)

Memory leakage is around 1-3 MB/s: input stream res is 720p50fps: do not happen with gstreamer 


Valgrind (--tool=massif)

ms_print massif.out.pid report:

    MB
138.0^                                                                       #
     |                                                                    @:@#
     |                                                                @@@:@:@#
     |                                                             @@@@@ :@:@#
     |                                                         @@::@@ @@ :@:@#
     |                                                    @::::@ : @@ @@ :@:@#
     |                                                ::::@: ::@ : @@ @@ :@:@#
     |                                             :::: : @: ::@ : @@ @@ :@:@#
     |                                         @::::::: : @: ::@ : @@ @@ :@:@#
     |                                     :@@:@:: :::: : @: ::@ : @@ @@ :@:@#
     |                                 @@:::@ :@:: :::: : @: ::@ : @@ @@ :@:@#
     |                             @@::@ : :@ :@:: :::: : @: ::@ : @@ @@ :@:@#
     |                          :::@ ::@ : :@ :@:: :::: : @: ::@ : @@ @@ :@:@#
     |                       ::::: @ ::@ : :@ :@:: :::: : @: ::@ : @@ @@ :@:@#
     |                  ::::::: :: @ ::@ : :@ :@:: :::: : @: ::@ : @@ @@ :@:@#
     |              @::::: : :: :: @ ::@ : :@ :@:: :::: : @: ::@ : @@ @@ :@:@#
     |        @  :::@:: :: : :: :: @ ::@ : :@ :@:: :::: : @: ::@ : @@ @@ :@:@#
     |     :@@@:::: @:: :: : :: :: @ ::@ : :@ :@:: :::: : @: ::@ : @@ @@ :@:@#
     | :::::@ @: :: @:: :: : :: :: @ ::@ : :@ :@:: :::: : @: ::@ : @@ @@ :@:@#
     | : : :@ @: :: @:: :: : :: :: @ ::@ : :@ :@:: :::: : @: ::@ : @@ @@ :@:@#
   0 +----------------------------------------------------------------------->Gi
     0                                                                   44.40

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
 52 47,650,320,026      143,592,272      137,496,744     6,095,528            0
 53 47,675,824,211      144,695,208      138,597,208     6,098,000            0
95.79% (138,597,208B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->73.56% (106,431,164B) 0xD241E83: i965_create_buffer_internal (i965_drv_video.c:2525)
| ->73.56% (106,431,164B) 0xD24206D: i965_CreateBuffer (i965_drv_video.c:2549)
|   ->73.56% (106,431,164B) 0x5238961: vaCreateBuffer (va.c:1058)
|     ->61.98% (89,678,400B) 0xE160F1: ??? (in /home/hsyrja/test/ffmpeg/ffmpeg)
|     | ->61.98% (89,678,400B) 0xE16594: ??? (in /home/hsyrja/test/ffmpeg/ffmpeg)
|     |   ->61.98% (89,676,300B) 0xE16845: ??? (in /home/hsyrja/test/ffmpeg/ffmpeg)
|     |   | ->61.98% (89,676,300B) 0xA64E81: ??? (in /home/hsyrja/test/ffmpeg/ffmpeg)
|     |   |   ->61.98% (89,676,300B) 0x493E59: ??? (in /home/hsyrja/test/ffmpeg/ffmpeg)
|     |   |     ->61.98% (89,676,300B) 0x498D9D: ??? (in /home/hsyrja/test/ffmpeg/ffmpeg)
|     |   |       ->61.98% (89,676,300B) 0x49A971: ??? (in /home/hsyrja/test/ffmpeg/ffmpeg)
|     |   |         ->61.98% (89,676,300B) 0x47D2AA: main (in /home/hsyrja/test/ffmpeg/ffmpeg)
 

[Carl Eugen Hoyos]

If you want to report a leak, please provide valgrind output, don't forget to recompile with debug information (this is the default).

[hsyrja]

Valgrind --tool=massif log

[hsyrja]

Valgrind --tool-massif report attached

[jkqxz]

The leak of VA buffers is due to a difference between the i965 driver and the VAAPI specification. See the descriptions around <​https://cgit.freedesktop.org/libva/tree/va/va.h#n2262> and <​https://cgit.freedesktop.org/libva/tree/va/va.h#n2390>, but the i965 driver does not free buffers after they are passed to vaRenderPicture().

This is worked around in <​https://git.libav.org/?p=libav.git;a=commit;h=4926fa9a4aa03f3b751f52e900b9efb87fea0591> along with <​https://git.libav.org/?p=libav.git;a=commit;h=221ffca6314ed3ba9d38464ea50cd85251c04e74> for the encoder (both changes pending merge) by detecting the i965 driver and having different behaviour in that case.

[hsyrja]

Thx submitted a bug report to Intel libva guys hope they will fix the issue. Found out that gstreamer do track and frees buffers manualy so no wonder why it won't have the same behavior.

[hsyrja]

​https://bugs.freedesktop.org/show_bug.cgi?id=97970

[jkqxz]

Workaround merged as a8d51bb42474b1641f45b5b3815864ea323a3e59 and 2dee500f4cbf64c547a37046e95141b84c85ee55.