[Android] MediaCodec h264 decoder is much slower than native h264 decoder

[bubbleguuum]

I've been benchmarking MediaCodec decoder performance for various codecs and although most MediaCodec decoders are faster than native, the h264_mediacodec decoder is noticeably slower, as in 2 to 3 times slower.

I'm using a custom FFmpeg 6.1 Android build, benchmark realized on a Google Pixel 4a
running latest Android 13

Benchmark result on a test h264 video with characteristics:

Duration: 00:09:56.50, start: 0.000000, bitrate: 4421 kb/s
Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 4087 kb/s, 24 fps, 24 tbr, 90k tbn (default)

With native h264 decoding:

% ffmpeg -benchmark -i /sdcard/Movies/bigbuckbunny1080p.mp4 -an -f null -
...
[out#0/null @ 0xb4000075e4c27a90] video:6710kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
frame=14315 fps=216 q=-0.0 Lsize=N/A time=00:09:56.41 bitrate=N/A speed=8.99x    
bench: utime=375.794s stime=10.060s rtime=66.314s
bench: maxrss=131580kB

With h264_mediacodec decoder:

% ./ffmpeg -benchmark -c:v h264_mediacodec -i /sdcard/Movies/bigbuckbunny1080p.mp4   -an -f null -
...
[out#0/null @ 0xb400007530d54f90] video:6710kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
frame=14315 fps=104 q=-0.0 Lsize=N/A time=00:09:56.41 bitrate=N/A speed=4.33x    
bench: utime=60.108s stime=23.083s rtime=137.887s
bench: maxrss=132824kB

So 8.99x speed for native vs 4.33x for h264_mediacodec.
Also, it seems the lower the resolution of the input video, the wider the gap is: on a 1024x436 video I observe a x3 difference!

Seems surprising there is such a gap in performance, especially since other decoders are faster (except also vp9_mediacodec which is a tiny bit slower than native).

Maybe it is due to the h264 native decoder being super optimized, but I thought I would report it for reference.

[quinkblack]

Firstly, decoding to Surface is more efficient, e.g.,

ffmpeg -benchmark -hwaccel mediacodec -init_hw_device mediacodec=mc,create_window=1 -t 60 -i /sdcard/Movies/bunny.mp4 -an  -f null -

frame= 1800 fps=191 q=-0.0 Lsize=N/A time=00:01:00.03 bitrate=N/A speed=6.37x    

bench: utime=4.116s stime=5.333s rtime=9.553s

Decoding to buffer is slow (not much overhead with my device, but it can be dramatically slow on some devcie)

ffmpeg -benchmark -hwaccel mediacodec  -t 60 -i /sdcard/Movies/bunny.mp4 -an  -f null -
frame= 1800 fps=152 q=-0.0 Lsize=N/A time=00:01:00.03 bitrate=N/A speed=5.06x     speed=2.01x    

bench: utime=7.850s stime=6.942s rtime=11.905s

Secondly, H.264 doesn't take much CPU to decoding. So with 8 CPU cores, software decoder can get a higher FPS, at the cost of high power consumption and device heating.

H.265 requires higher computational power, so it's easy for hardware decoder to overtake software decoder.

[bubbleguuum]

Thank you for this information.

On my Pixel 4a, performance is close (slightly worse for Surface) between Surface and buffer (4.04x vs 4.16x) for the bbb video whose specs are in the first post.
On which Android device did you perform this benchmark ?

In any case, it does not really matter because as you mentioned, h264_mediacodec is much less CPU intensive, which is a characteristic that I value more for my use case (as long as it is fast enough for realtime transcode) than raw decoding speed.