Opened 2 years ago

Last modified 18 months ago

#8377 new defect

Using kmsgrab + h264_vaapi with audio causes stutter

Reported by: Asineth Owned by:
Priority: normal Component: undetermined
Version: git-master Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Whenever I use kmsgrab + vaapi to record my desktop with ffmpeg and I include audio (I've tested this with both -f pulse and -f alsa) it causes the video to become stuttery. I included commands, ffmpeg logs, and the videos to compare.

Distro: Archlinux x86_64

Here's my vainfo:
vainfo: VA-API version: 1.5 (libva 2.5.0)
vainfo: Driver version: Mesa Gallium driver 19.2.3 for AMD Radeon (TM) RX 470 Graphics (POLARIS10, DRM 3.33.0, 5.3.8-arch1-1, LLVM 9.0.0)
vainfo: Supported profile and entrypoints

VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileHEVCMain : VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointEncSlice
VAProfileHEVCMain10 : VAEntrypointVLD
VAProfileJPEGBaseline : VAEntrypointVLD
VAProfileNone : VAEntrypointVideoProc

# without audio
# log: https://www.transfer.sh/KfuPb/ffmpeg-20191110-172753.log
# video: https://streamable.com/eoyp2
ffmpeg -y -framerate 60 -vsync 0 -device /dev/dri/card0 -vaapi_device /dev/dri/renderD128 -f kmsgrab -i - -vf 'hwmap,scale_vaapi=format=nv12' -c:v h264_vaapi -profile:v constrained_baseline -bf 0 output.mp4 -loglevel debug -report
# with audio
# log: https://www.transfer.sh/cVdJb/ffmpeg-20191110-172632.log
# video: https://streamable.com/vaxin
ffmpeg -y -framerate 60 -vsync 0 -device /dev/dri/card0 -vaapi_device /dev/dri/renderD128 -f kmsgrab -i - -f alsa -i default -vf 'hwmap,scale_vaapi=format=nv12' -c:v h264_vaapi -c:a aac -profile:v constrained_baseline -bf 0 output_waudio.mp4 -loglevel debug -report

Attachments (2)

output.mp4 (1.3 MB ) - added by 0x53ee71ebe11e 18 months ago.
2 seconds of moving around in a game demostrating the video stuttering
ffmpeglog.txt (249.5 KB ) - added by 0x53ee71ebe11e 18 months ago.
ffmpeg debug output

Download all attachments as: .zip

Change History (7)

in reply to:  description ; comment:1 by Carl Eugen Hoyos, 2 years ago

Replying to asineth:

Whenever I use kmsgrab + vaapi to record my desktop with ffmpeg and I include audio (I've tested this with both -f pulse and -f alsa) it causes the video to become stuttery.
I included commands, ffmpeg logs, and the videos to compare.

Sadly not.

in reply to:  1 comment:2 by Asineth, 2 years ago

Replying to cehoyos:

Replying to asineth:

Whenever I use kmsgrab + vaapi to record my desktop with ffmpeg and I include audio (I've tested this with both -f pulse and -f alsa) it causes the video to become stuttery.
I included commands, ffmpeg logs, and the videos to compare.

Sadly not.

I included the commands, and links to the logs and videos in the post.

comment:3 by Carl Eugen Hoyos, 2 years ago

Please post the command line you tested together with the complete, uncut console output here in the ticket to make it valid.

comment:4 by Asineth, 2 years ago

Here's my cmdline for where I started.

ffmpeg -y -framerate 60 -vsync 0 -device /dev/dri/card0 -vaapi_device /dev/dri/renderD128 -f kmsgrab -i - -vf 'hwmap,scale_vaapi=format=nv12' -c:v h264_vaapi -profile:v constrained_baseline -bf 0 output.mp4 -loglevel debug -report

This produces a smooth recording. I've attached the output and log as output1.mp4 and output1.log

I added in audio with "-f alsa -i default -c:a aac" and this is my new cmdline.

ffmpeg -y -framerate 60 -vsync 0 -device /dev/dri/card0 -vaapi_device /dev/dri/renderD128 -f kmsgrab -i - -f alsa -i default -c:a aac -vf 'hwmap,scale_vaapi=format=nv12' -c:v h264_vaapi -profile:v constrained_baseline -bf 0 output_waudio.mp4 -loglevel debug -report

This produces a stuttery and unuseable recording. I've attached the output and log as output2.mp4 and output2.log

What I don't understand is: why does the video become stuttery and unuseable when I add in audio? Am I missing something or doing something wrong?

Files (too large, ~20MB for the videos):
output1.mp4: https://anonfile.com/Z4Mf4aA2n2/output1_mp4
output1.log: https://anonfile.com/Q5M442A0n0/output1_log
output2.mp4: https://anonfile.com/WbMf4eA2n3/output2_mp4
output2.log: https://anonfile.com/O1M943A4n4/output2_log

by 0x53ee71ebe11e, 18 months ago

Attachment: output.mp4 added

2 seconds of moving around in a game demostrating the video stuttering

by 0x53ee71ebe11e, 18 months ago

Attachment: ffmpeglog.txt added

ffmpeg debug output

comment:5 by 0x53ee71ebe11e, 18 months ago

Version: unspecifiedgit-master

I have the very same problem. I added a short video demonstrating the stuttering. If you step through the video frame by frame, you can see that the frames are in an incorrect order. The stuttering only happens when also recording audio.

Ubuntu 19.10
kernel 5.6.7-050607-generic
gpu: RX 5600 XT
gpu drivers from https://launchpad.net/~oibaf/+archive/ubuntu/graphics-drivers
ffmpeg from git master

$ vainfo
libva info: VA-API version 1.5.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so
libva info: Found init function vaDriverInit_1_5
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.5 (libva 2.5.0)
vainfo: Driver version: Mesa Gallium driver 20.2.0-devel for AMD Radeon RX 5600 XT (NAVI10, DRM 3.36.0, 5.6.7-050607-generic, LLVM 9.0.1)
vainfo: Supported profile and entrypoints

VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileHEVCMain : VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointEncSlice
VAProfileHEVCMain10 : VAEntrypointVLD
VAProfileHEVCMain10 : VAEntrypointEncSlice
VAProfileJPEGBaseline : VAEntrypointVLD
VAProfileVP9Profile0 : VAEntrypointVLD
VAProfileVP9Profile2 : VAEntrypointVLD
VAProfileNone : VAEntrypointVideoProc

ffmpeg command:

$HOME/ffmpeg/ffmpeg \
-v 9 -loglevel 99 \
-threads 8 \
-thread_queue_size 512 \
-f kmsgrab -framerate 59.4 -i - \
-f pulse -i default \
-map 0 \
-map 1 \
-vf 'hwmap=derive_device=vaapi,scale_vaapi=1920:1080:nv12' -c:v h264_vaapi -qp 40 \
-c:a libmp3lame -ab 128k \
-t 2 \
/tmp/output.mp4 >/tmp/ffmpeglog.txt 2>&1

video and debug output added as attachment

Also this doesn't seem to be specific to h264_vaapi. Using hevc_vaapi instead leads to the exact same problem.

Last edited 18 months ago by 0x53ee71ebe11e (previous) (diff)
Note: See TracTickets for help on using tickets.