FFmpeg VAAPI hwaccel decode -> hwupload/hwdownload -> x264 will block

[Jun Zhao]

Summary of the bug:
How to reproduce:

% ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -i input.mp4 -vf 'format=nv12|vaapi,hwupload,scale_vaapi=w=1280:h=720:format=yuv420p,hwdownload' -c:v libx264 output.mp4
ffmpeg version : master branch (f1baafa)

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

[jkqxz]

I've seen this too a few times - seems to be something in libavfilter to do with format negotiation around the hw* filters, but I've never pursued it beyond that. You can work around it by specifying the formats explicitly (i.e. add 'format=yuv420p' to the end of your filter graph).

[Jun Zhao]

The work around is working, tks, Mark. BTW: libav is ok in this case.

[jkqxz]

The hang is fixed by 8a442d7a8a687a469ca502a18a0c68f5302b15e0. The command line above works sensibly with that, but there are still cases where additional format filters are necessary to stop it making bad choices (though they do at least fail immediately now rather than hanging).