MediaCodec Encoder: Gibberish output produced on emulators (generally below < API 31)

[Ajay Shrestha]

What you were trying to accomplish?
I am trying to transcode an mp4 file using the mediacodec encoder support recently added by Zhao Zhili here (​https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/0ff18a7d6d496f89d9e007ddd1cad7116baf5c7c)

The problem you encountered:
Currently, only on emulators with OS API levels less than API 31, The output video is simply gibberish with a repetitive pattern of red, purple, or random colors

The exact command line you were using:
-hwaccel mediacodec -hwaccel_output_format mediacodec -i "/data/user/0/com.test.ffmpeg_kit_android_demo/cache/test_input.mp4" -c:v h264_mediacodec -ndk_codec 1 -y "/data/user/0/com.test.ffmpeg_kit_android_demo/cache/test_output.mp4" -loglevel trace

Note: This is built and run using FFMPEG-Kit

[quinkblack]

I don't know about FFmpeg-Kit, especially why encoder has yuv420p pixel format while -hwaccel_output_format mediacodec.

Firstly, try if pix_fmt nv12 works: -c:v h264_mediacodec -ndk_codec 1 -pix_fmt nv12.

Secondly, try if -c:v h264_mediacodec -i test.mp4 -c:v h264_mediacodec -ndk_codec 1 -pix_fmt nv12 works, which means decoding to buffer and feed it to encoder, rather than decoding to surface.

I have some pending patches, but unlikely related to this ticket.

​http://ffmpeg.org/pipermail/ffmpeg-devel/2022-December/304671.html
patch 2 3/7 has been replaced by patch v3 3/7.
​http://ffmpeg.org/pipermail/ffmpeg-devel/2022-December/304717.html

[Ajay Shrestha]

Replying to quinkblack:

I don't know about FFmpeg-Kit, especially why encoder has yuv420p pixel format while -hwaccel_output_format mediacodec.

Firstly, try if pix_fmt nv12 works: -c:v h264_mediacodec -ndk_codec 1 -pix_fmt nv12.

Secondly, try if -c:v h264_mediacodec -i test.mp4 -c:v h264_mediacodec -ndk_codec 1 -pix_fmt nv12 works, which means decoding to buffer and feed it to encoder, rather than decoding to surface.

I have some pending patches, but unlikely related to this ticket.

​http://ffmpeg.org/pipermail/ffmpeg-devel/2022-December/304671.html
patch 2 3/7 has been replaced by patch v3 3/7.
​http://ffmpeg.org/pipermail/ffmpeg-devel/2022-December/304717.html

1.I tried with -pxfmt nv12, and got the same gibberish result
2.Got the error Error while opening decoder for input stream when trying to decoding to buffer

I have attached the console log for both of them (console_output_nv12.log and console_output_decoding_to_buffer.log)

[quinkblack]

The device is an emulator. I think there is no real hardware decoder/encoder on the emulator, only software implementation. FFmpeg mediacodec wrapper via JNI skip software decoders explicitly, so decoder doesn't work.

You can try the NDK implementation, put an '-ndk_codec 1' before -i

-ndk_codec 1 -c:v h264_mediacodec -i

I guess it will work. It's only for test. There's no sense to use MediaCodec software decoder, since FFmpeg's software decoder do better, that's

ffmpeg -i test.mp4 -c:v h264_mediacodec out.mp4

Should work. There is no real mediacodec hardware encoder neither. The software implementition has YUV420P support, so -pix_fmt nv12 is unnecessary.

[Ajay Shrestha]

Replying to quinkblack:

The device is an emulator. I think there is no real hardware decoder/encoder on the emulator, only software implementation. FFmpeg mediacodec wrapper via JNI skip software decoders explicitly, so decoder doesn't work.

You can try the NDK implementation, put an '-ndk_codec 1' before -i

-ndk_codec 1 -c:v h264_mediacodec -i

I guess it will work. It's only for test. There's no sense to use MediaCodec software decoder, since FFmpeg's software decoder do better, that's

ffmpeg -i test.mp4 -c:v h264_mediacodec out.mp4

Should work. There is no real mediacodec hardware encoder neither. The software implementition has YUV420P support, so -pix_fmt nv12 is unnecessary.

Still getting the same Gibberish output with both options. Attached the log below

[quinkblack]

I have tested with ffmpeg cmd on an "Pixel 4 API 30 x64" emulator and got no problem. Sorry I can't reproduce the issue.

FFmpeg cmd:

​https://drive.google.com/file/d/1wuIvcdHVuwdVM-83qt0PG_5FA1GNxJhl/view?usp=sharing

[quinkblack]

By the way, Android system log is very helpful when debug mediacodec.

[Ajay Shrestha]

[​​https://drive.google.com/file/d/1wuIvcdHVuwdVM-83qt0PG_5FA1GNxJhl/view?usp=sharing]
I couldn't open this file. It has some kind of binary string. Could you please share again ?

[Balling]

Replying to Ajay Shrestha:

[​​https://drive.google.com/file/d/1wuIvcdHVuwdVM-83qt0PG_5FA1GNxJhl/view?usp=sharing]
I couldn't open this file. It has some kind of binary string. Could you please share again ?

That is an executable.