kmsgrab not working on AMD cards

[tildearrow]

Description:
kmsgrab seems to not work on AMD cards, or at least the one I tested with.

Steps to reproduce:

$ sudo ./ffmpeg -report -f kmsgrab -device /dev/dri/card1 -i - -vf 'hwdownload,format=bgr0' -c:v rawvideo -f avi -y /dev/null
ffmpeg started on 2018-08-22 at 23:39:54
Report written to "ffmpeg-20180822-233954.log"
ffmpeg version N-91666-g70d2bab80b Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 8.2.0 (GCC)
  configuration: --disable-everything --enable-gpl --enable-libdrm --enable-indev=kmsgrab --enable-decoder=wrapped_avframe --enable-encoder=rawvideo --enable-filter=format --enable-filter=hwdownload --enable-protocol=file --enable-muxer=avi
  libavutil      56. 19.100 / 56. 19.100
  libavcodec     58. 24.101 / 58. 24.101
  libavformat    58. 17.103 / 58. 17.103
  libavdevice    58.  4.101 / 58.  4.101
  libavfilter     7. 26.100 /  7. 26.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
[kmsgrab @ 0x5557f5102a00] Using plane 41 to locate framebuffers.
[kmsgrab @ 0x5557f5102a00] Template framebuffer is 95: 3840x2160 32bpp 24b depth.
Input #0, kmsgrab, from 'pipe:':
  Duration: N/A, start: 1534999194.427423, bitrate: N/A
    Stream #0:0: Video: wrapped_avframe, drm_prime, 3840x2160, 30 tbr, 1000k tbn, 1000k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (wrapped_avframe (native) -> rawvideo (native))
[AVHWFramesContext @ 0x5557f5105980] Failed to map DRM object 5 to memory: 1.
[hwdownload @ 0x5557f5113900] Failed to download frame: -1.
Error while filtering: Operation not permitted
Failed to inject frame into filter network: Operation not permitted
Error while processing the decoded data for stream #0:0
Conversion failed!

Additional information:
Attached FFmpeg log.

No extra messages in the kernel log have been generated.

This may actually be a problem in amdgpu rather than FFmpeg, but I am not sure.

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

System specifications:

CPU:       Quad core Intel Core i7-6700K (-MT-MCP-) cache: 8192 KB
           clock speeds: max: 4001 MHz 1: 4050 MHz 2: 4025 MHz 3: 4000 MHz 4: 4095 MHz 5: 4001 MHz
           6: 4004 MHz 7: 4063 MHz 8: 4015 MHz
Graphics:  Card-1: Intel HD Graphics 530
           Card-2: Advanced Micro Devices [AMD/ATI] Vega 10 XTX [Radeon Vega Frontier Edition]
           Display Server: x11 (X.Org 1.20.0 ) driver: modesetting Resolution: 3840x2160
           OpenGL: renderer: Radeon Vega Frontier Edition (VEGA10, DRM 3.26.0, 4.18.3-zen1-1-zen, LLVM 6.0.1)
           version: 4.5 Mesa 18.1.6

uname -a:

Linux linux 4.18.3-zen1-1-zen #1 ZEN SMP PREEMPT Sat Aug 18 09:23:50 UTC 2018 x86_64 GNU/Linux

[jkqxz]

DRM objects used for scanout need not be CPU-mappable, so there is no guarantee that kmsgrab output will be downloadable directly as you are trying to do here.

On AMD external GPUs it should be mappable to VAAPI, and then can be downloaded from there - e.g. something like:

ffmpeg -device /dev/dri/card1 -f kmsgrab -i - -vf 'hwmap=derive_device=vaapi,hwdownload,format=bgr0' -c:v libx264 out.mp4