Opened 14 months ago

Last modified 14 months ago

#9280 new defect

meson-vdec: poor playback of h264

Reported by: danboid Owned by:
Priority: normal Component: undetermined
Version: unspecified Keywords: v4l2m2m
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description (last modified by danboid)

Summary of the bug:

I am experiencing very jerky playback using ffplay under Manjaro ARM unstable to play a UHD h264 mp4 on a X96 Air Q1000 TV box which is based on the Amlogic S905X3 SoC w/ meson-vdec VPU. I get 100's of dropped frames in under 30 seconds.

How to reproduce:

I am trying to play this video:

https://drive.google.com/file/d/14z1Gd8zXbJ2XWvULvmWXeqLVRcgUHo2p/view?usp=sharing

Using this command under Wayland GNOME:

$ ffplay -vcodec h264_v4l2m2m VID_20210512_180639.mp4 
ffplay version n4.4 Copyright (c) 2003-2021 the FFmpeg developers
  built with gcc 10.2.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-shared --enable-version3 --host-cflags='"-fPIC"'
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'VID_20210512_180639.mp4':  
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2021-05-12T17:07:00.000000Z
    com.android.version: 10
  Duration: 00:00:20.03, start: 0.000000, bitrate: 43964 kb/s
  Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, unknown/bt470bg/unknown), 3840x2160, 43449 kb/s, SAR 1:1 DAR 16:9, 30 fps, 30 tbr, 90k tbn, 180k tbc (default)
    Metadata:
      creation_time   : 2021-05-12T17:07:00.000000Z
      handler_name    : VideoHandle
      vendor_id       : [0][0][0][0]
  Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s (default)
    Metadata:
      creation_time   : 2021-05-12T17:07:00.000000Z
      handler_name    : SoundHandle
      vendor_id       : [0][0][0][0]
[h264_v4l2m2m @ 0xffff5c0fc000] Using device /dev/video0
[h264_v4l2m2m @ 0xffff5c0fc000] driver 'meson-vdec' on card 'Amlogic Video Decoder' in mplane mode
[h264_v4l2m2m @ 0xffff5c0fc000] requesting formats: output=H264 capture=NM12
 176.91 A-V: 71.125 fd= 501 aq=    0KB vq=    0KB sq=    0B f=0/0   

I can upload a video of the playback if required but it plays pretty much exactly the same as it does under mpv as shown here:

https://github.com/mpv-player/mpv/issues/8884#

I'm a running Linux kernel 5.12.1-1-MANJARO-ARM

Change History (13)

comment:1 by danboid, 14 months ago

Description: modified (diff)

comment:2 by Carl Eugen Hoyos, 14 months ago

Component: ffplayundetermined
Keywords: meson-vdec removed

Please test current FFmpeg git head, the only version supported here, and please test ffmpeg instead of ffplay, there is generally no hardware acceleration with ffplay.

comment:3 by danboid, 14 months ago

Hi Carl

OK, I will build git head and report back.

What ffmpeg command should I use to test this?

Thanks

comment:4 by danboid, 14 months ago

ffmpeg git is building atm.

I'll try a command like this, unless I'm advised otherwise:

./ffmpeg -hwaccel drm -hwaccel_output_format drm_prime -c:v h264_v4l2m2m -i VID_20210512_180639.mp4 -map 0:v -f null -y /dev/null

comment:5 by danboid, 14 months ago

I successfully built and installed ffmpeg git under Manjaro ARM and I can run this command without any errors:

$ ffmpeg -c:v h264_v4l2m2m -i VID_20210512_180639.mp4 -f null -

As per the example given here:

https://lwn.net/Articles/767126/

How might I modify that ffmpeg command to output a picture (ie to play the video under Wayland) and if decoding is working fine under ffmpeg, why does it not work under ffplay?

Last edited 14 months ago by danboid (previous) (diff)

comment:6 by danboid, 14 months ago

This LKML post details a more recent update of the meson driver ie adding of h264 decoding support:

https://lwn.net/Articles/767126/

I have emailed Mauro, who seems to be the most recent maintainer/updater of the meson code, to see if he has any suggestions.

Last edited 14 months ago by danboid (previous) (diff)

in reply to:  5 comment:7 by Balling, 14 months ago

Replying to danboid:

How might I modify that ffmpeg command to output a picture (ie to play the video under Wayland) and if decoding is working fine under ffmpeg, why does it not work under ffplay?

Very simple. ffmpeg -i INPUT -f opengl "window title"

This will use SDL, so no cool colorspaces and many bugs.

comment:8 by danboid, 14 months ago

Hi balling

My GPU (Mali G31) doesn't support OpenGL, only OpenGLES 3.x but I thought I'd try -f opengl anyway and it didn't work. I get the error:

"Requested output format 'opengl' is not a suitable output format"

I've not looked at the ffmpeg source but there doesn't seem to be much info in the ffmpeg man page about -f and its settings.

comment:9 by Balling, 14 months ago

man page about -f and its settings.

ffmpeg -formats

ffmpeg -sample_fmts

comment:10 by danboid, 14 months ago

Thanks balling

I have got a bit closer to playing to the test file with ffmpeg using the sdl2 demuxer but it says

"Unsupported pixel format 'nv12'"

$ ffmpeg -c:v h264_v4l2m2m -i VID_20210512_180639.mp4 -f sdl2 -
ffmpeg version N-102655-g282b9f4cba Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10.2.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
  libavutil      57.  0.100 / 57.  0.100
  libavcodec     59.  1.101 / 59.  1.101
  libavformat    59.  2.101 / 59.  2.101
  libavdevice    59.  0.100 / 59.  0.100
  libavfilter     8.  0.101 /  8.  0.101
  libswscale      6.  0.100 /  6.  0.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'VID_20210512_180639.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2021-05-12T17:07:00.000000Z
    com.android.version: 10
  Duration: 00:00:20.03, start: 0.000000, bitrate: 43964 kb/s
  Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, unknown/bt470bg/unknown), 3840x2160, 43449 kb/s, SAR 1:1 DAR 16:9, 30 fps, 30 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2021-05-12T17:07:00.000000Z
      handler_name    : VideoHandle
      vendor_id       : [0][0][0][0]
  Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s (default)
    Metadata:
      creation_time   : 2021-05-12T17:07:00.000000Z
      handler_name    : SoundHandle
      vendor_id       : [0][0][0][0]
[h264_v4l2m2m @ 0xaaab223bf530] Using device /dev/video0
[h264_v4l2m2m @ 0xaaab223bf530] driver 'meson-vdec' on card 'Amlogic Video Decoder' in mplane mode
[h264_v4l2m2m @ 0xaaab223bf530] requesting formats: output=H264 capture=NM12
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_v4l2m2m) -> rawvideo (native))
Press [q] to stop, [?] for help
[sdl,sdl2 @ 0xaaab22384f50] Unsupported pixel format 'nv12'.
Output #0, sdl,sdl2, to 'pipe:':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    com.android.version: 10
    encoder         : Lavf59.2.101
  Stream #0:0(eng): Video: rawvideo (NV12 / 0x3231564E), nv12(progressive), 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 2985984 kb/s, 30 fps, 30 tbn (default)
    Metadata:
      creation_time   : 2021-05-12T17:07:00.000000Z
      handler_name    : VideoHandle
      vendor_id       : [0][0][0][0]
      encoder         : Lavc59.1.101 rawvideo
Unsupported pixel format
av_interleaved_write_frame(): Operation not permitted
frame=    1 fps=0.8 q=-0.0 Lsize=N/A time=00:00:00.03 bitrate=N/A speed=0.0271x    
video:12150kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!

I said previously that I didn't think my gpu supported opengl but when using -loglevel verbose that is apparently what ffplay is using to play the video although using -f opengl - doesn't work with ffmpeg. It says

"Requested output format 'opengl' is not a suitable output format"

$ ffplay -loglevel verbose -vcodec h264_v4l2m2m VID_20210512_180639.mp4 
ffplay version N-102655-g282b9f4cba Copyright (c) 2003-2021 the FFmpeg developers
  built with gcc 10.2.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
  libavutil      57.  0.100 / 57.  0.100
  libavcodec     59.  1.101 / 59.  1.101
  libavformat    59.  2.101 / 59.  2.101
  libavdevice    59.  0.100 / 59.  0.100
  libavfilter     8.  0.101 /  8.  0.101
  libswscale      6.  0.100 /  6.  0.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Initialized opengl renderer.
[h264 @ 0xffff74001e90] Reinit context to 3840x2160, pix_fmt: yuv420p
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'VID_20210512_180639.mp4':  
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2021-05-12T17:07:00.000000Z
    com.android.version: 10
  Duration: 00:00:20.03, start: 0.000000, bitrate: 43964 kb/s
  Stream #0:0(eng): Video: h264 (High), 1 reference frame (avc1 / 0x31637661), yuv420p(tv, unknown/bt470bg/unknown, left), 3840x2160, 43449 kb/s, SAR 1:1 DAR 16:9, 30 fps, 30 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2021-05-12T17:07:00.000000Z
      handler_name    : VideoHandle
      vendor_id       : [0][0][0][0]
  Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s (default)
    Metadata:
      creation_time   : 2021-05-12T17:07:00.000000Z
      handler_name    : SoundHandle
      vendor_id       : [0][0][0][0]
[ffplay_abuffer @ 0xffff741000a0] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:0x3
[ffplay_abuffersink @ 0xffff740ed010] auto-inserting filter 'auto_resampler_0' between the filter 'ffplay_abuffer' and the filter 'ffplay_abuffersink'
[auto_resampler_0 @ 0xffff74011460] ch:2 chl:stereo fmt:fltp r:48000Hz -> ch:2 chl:stereo fmt:s16 r:48000Hz
[h264_v4l2m2m @ 0xffff740f9440] Using device /dev/video0
[h264_v4l2m2m @ 0xffff740f9440] driver 'meson-vdec' on card 'Amlogic Video Decoder' in mplane mode
[h264_v4l2m2m @ 0xffff740f9440] requesting formats: output=H264 capture=NM12
[ffplay_abuffer @ 0xffff740ed010] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:0x3
[ffplay_abuffersink @ 0xffff74011460] auto-inserting filter 'auto_resampler_0' between the filter 'ffplay_abuffer' and the filter 'ffplay_abuffersink'
[auto_resampler_0 @ 0xffff741000a0] ch:2 chl:stereo fmt:fltp r:48000Hz -> ch:2 chl:stereo fmt:s16 r:48000Hz
[ffplay_buffer @ 0xffff540321f0] w:3840 h:2160 pixfmt:nv12 tb:1/90000 fr:30/1 sar:0/1
[auto_scaler_0 @ 0xffff54033650] w:iw h:ih flags:'bicubic' interl:0
[ffplay_buffersink @ 0xffff54032830] auto-inserting filter 'auto_scaler_0' between the filter 'ffplay_buffer' and the filter 'ffplay_buffersink'
[auto_scaler_0 @ 0xffff54033650] w:3840 h:2160 fmt:nv12 sar:0/1 -> w:3840 h:2160 fmt:yuv420p sar:0/1 flags:0x4
    Last message repeated 2 times
[auto_scaler_0 @ 0xffff54033650] w:3840 h:2160 fmt:nv12 sar:1/1 -> w:3840 h:2160 fmt:yuv420p sar:1/1 flags:0x4
Created 3840x2160 texture with SDL_PIXELFORMAT_IYUV.=    0B f=0/0   
[AVIOContext @ 0xffff74009340] Statistics: 109381145 bytes read, 5 seeks
Last edited 14 months ago by danboid (previous) (diff)

comment:11 by danboid, 14 months ago

Is ffplay actually using opengl or is it really using opengl via SDL? I am referring to the line

Created 3840x2160 texture with SDL_PIXELFORMAT_IYUV.=    0B f=0/0

From my last ffplay log which indicates to me that SDL does actually support the required pixel format but ffmpeg is detecting it incorrectly, maybe? Although it plays very jerkily, the colours are displayed correctly under ffplay 'opengl' output.

comment:13 by danboid, 14 months ago

Slightly OT but I have written a wiki page for installing and running Manjaro on Amlogic TV boxes

https://wiki.manjaro.org/index.php/Amlogic_TV_boxes

Note: See TracTickets for help on using tickets.