Using kmsgrab + h264_vaapi with audio causes stutter

[Asineth]

Whenever I use kmsgrab + vaapi to record my desktop with ffmpeg and I include audio (I've tested this with both -f pulse and -f alsa) it causes the video to become stuttery. I included commands, ffmpeg logs, and the videos to compare.

Distro: Archlinux x86_64

Here's my vainfo:
vainfo: VA-API version: 1.5 (libva 2.5.0)
vainfo: Driver version: Mesa Gallium driver 19.2.3 for AMD Radeon (TM) RX 470 Graphics (POLARIS10, DRM 3.33.0, 5.3.8-arch1-1, LLVM 9.0.0)
vainfo: Supported profile and entrypoints

VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileHEVCMain : VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointEncSlice
VAProfileHEVCMain10 : VAEntrypointVLD
VAProfileJPEGBaseline : VAEntrypointVLD
VAProfileNone : VAEntrypointVideoProc

# without audio
# log: ​https://www.transfer.sh/KfuPb/ffmpeg-20191110-172753.log
# video: ​https://streamable.com/eoyp2
ffmpeg -y -framerate 60 -vsync 0 -device /dev/dri/card0 -vaapi_device /dev/dri/renderD128 -f kmsgrab -i - -vf 'hwmap,scale_vaapi=format=nv12' -c:v h264_vaapi -profile:v constrained_baseline -bf 0 output.mp4 -loglevel debug -report
# with audio
# log: ​https://www.transfer.sh/cVdJb/ffmpeg-20191110-172632.log
# video: ​https://streamable.com/vaxin
ffmpeg -y -framerate 60 -vsync 0 -device /dev/dri/card0 -vaapi_device /dev/dri/renderD128 -f kmsgrab -i - -f alsa -i default -vf 'hwmap,scale_vaapi=format=nv12' -c:v h264_vaapi -c:a aac -profile:v constrained_baseline -bf 0 output_waudio.mp4 -loglevel debug -report

[Carl Eugen Hoyos]

Replying to asineth:

Whenever I use kmsgrab + vaapi to record my desktop with ffmpeg and I include audio (I've tested this with both -f pulse and -f alsa) it causes the video to become stuttery.
I included commands, ffmpeg logs, and the videos to compare.

Sadly not.

[Asineth]

Replying to cehoyos:

Replying to asineth:

Whenever I use kmsgrab + vaapi to record my desktop with ffmpeg and I include audio (I've tested this with both -f pulse and -f alsa) it causes the video to become stuttery.
I included commands, ffmpeg logs, and the videos to compare.

Sadly not.

I included the commands, and links to the logs and videos in the post.

[Carl Eugen Hoyos]

Please post the command line you tested together with the complete, uncut console output here in the ticket to make it valid.

[Asineth]

Here's my cmdline for where I started.

ffmpeg -y -framerate 60 -vsync 0 -device /dev/dri/card0 -vaapi_device /dev/dri/renderD128 -f kmsgrab -i - -vf 'hwmap,scale_vaapi=format=nv12' -c:v h264_vaapi -profile:v constrained_baseline -bf 0 output.mp4 -loglevel debug -report

This produces a smooth recording. I've attached the output and log as output1.mp4 and output1.log

I added in audio with "-f alsa -i default -c:a aac" and this is my new cmdline.

ffmpeg -y -framerate 60 -vsync 0 -device /dev/dri/card0 -vaapi_device /dev/dri/renderD128 -f kmsgrab -i - -f alsa -i default -c:a aac -vf 'hwmap,scale_vaapi=format=nv12' -c:v h264_vaapi -profile:v constrained_baseline -bf 0 output_waudio.mp4 -loglevel debug -report

This produces a stuttery and unuseable recording. I've attached the output and log as output2.mp4 and output2.log

What I don't understand is: why does the video become stuttery and unuseable when I add in audio? Am I missing something or doing something wrong?

Files (too large, ~20MB for the videos):
output1.mp4: ​https://anonfile.com/Z4Mf4aA2n2/output1_mp4
output1.log: ​https://anonfile.com/Q5M442A0n0/output1_log
output2.mp4: ​https://anonfile.com/WbMf4eA2n3/output2_mp4
output2.log: ​https://anonfile.com/O1M943A4n4/output2_log

[0x53ee71ebe11e]

2 seconds of moving around in a game demostrating the video stuttering

[0x53ee71ebe11e]

ffmpeg debug output

[0x53ee71ebe11e]

I have the very same problem. I added a short video demonstrating the stuttering. If you step through the video frame by frame, you can see that the frames are in an incorrect order. The stuttering only happens when also recording audio.

Ubuntu 19.10
kernel 5.6.7-050607-generic
gpu: RX 5600 XT
gpu drivers from ​https://launchpad.net/~oibaf/+archive/ubuntu/graphics-drivers
ffmpeg from git master

$ vainfo
libva info: VA-API version 1.5.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so
libva info: Found init function vaDriverInit_1_5
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.5 (libva 2.5.0)
vainfo: Driver version: Mesa Gallium driver 20.2.0-devel for AMD Radeon RX 5600 XT (NAVI10, DRM 3.36.0, 5.6.7-050607-generic, LLVM 9.0.1)
vainfo: Supported profile and entrypoints

VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileHEVCMain : VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointEncSlice
VAProfileHEVCMain10 : VAEntrypointVLD
VAProfileHEVCMain10 : VAEntrypointEncSlice
VAProfileJPEGBaseline : VAEntrypointVLD
VAProfileVP9Profile0 : VAEntrypointVLD
VAProfileVP9Profile2 : VAEntrypointVLD
VAProfileNone : VAEntrypointVideoProc

ffmpeg command:

$HOME/ffmpeg/ffmpeg \
-v 9 -loglevel 99 \
-threads 8 \
-thread_queue_size 512 \
-f kmsgrab -framerate 59.4 -i - \
-f pulse -i default \
-map 0 \
-map 1 \
-vf 'hwmap=derive_device=vaapi,scale_vaapi=1920:1080:nv12' -c:v h264_vaapi -qp 40 \
-c:a libmp3lame -ab 128k \
-t 2 \
/tmp/output.mp4 >/tmp/ffmpeglog.txt 2>&1

video and debug output added as attachment

Also this doesn't seem to be specific to h264_vaapi. Using hevc_vaapi instead leads to the exact same problem.

[rosehuds]

are there any updates on this, or any logs or info i can provide to help out? the issue is driving me up the wall