Opened 16 months ago

Closed 2 months ago

#10097 closed defect (worksforme)

MediaCodec Encoder: Gibberish output produced on emulators (generally below < API 31)

Reported by: Ajay Shrestha Owned by:
Priority: normal Component: undetermined
Version: git-master Keywords: mediacodec
Cc: Ajay Shrestha Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description (last modified by Ajay Shrestha)

What you were trying to accomplish?
I am trying to transcode an mp4 file using the mediacodec encoder support recently added by Zhao Zhili here (https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/0ff18a7d6d496f89d9e007ddd1cad7116baf5c7c)

The problem you encountered:
Currently, only on emulators with OS API levels less than API 31, The output video is simply gibberish with a repetitive pattern of red, purple, or random colors

The exact command line you were using:
-hwaccel mediacodec -hwaccel_output_format mediacodec -i "/data/user/0/com.test.ffmpeg_kit_android_demo/cache/test_input.mp4" -c:v h264_mediacodec -ndk_codec 1 -y "/data/user/0/com.test.ffmpeg_kit_android_demo/cache/test_output.mp4" -loglevel trace

Note: This is built and run using FFMPEG-Kit

Attachments (7)

console_output.log (17.4 KB ) - added by Ajay Shrestha 16 months ago.
test_input.mp4 (13.4 KB ) - added by Ajay Shrestha 16 months ago.
output_video.mp4 (454.3 KB ) - added by Ajay Shrestha 16 months ago.
console_output_nv12.log (18.8 KB ) - added by Ajay Shrestha 16 months ago.
console_output_decoding_to_buffer.log (9.8 KB ) - added by Ajay Shrestha 16 months ago.
console_log_ndk.log (19.6 KB ) - added by Ajay Shrestha 16 months ago.
console_log_2.log (21.3 KB ) - added by Ajay Shrestha 16 months ago.

Download all attachments as: .zip

Change History (18)

by Ajay Shrestha, 16 months ago

Attachment: console_output.log added

by Ajay Shrestha, 16 months ago

Attachment: test_input.mp4 added

by Ajay Shrestha, 16 months ago

Attachment: output_video.mp4 added

comment:1 by Ajay Shrestha, 16 months ago

Description: modified (diff)

comment:2 by Carl Eugen Hoyos, 16 months ago

Component: ffmpegundetermined
Version: unspecifiedgit-master

comment:3 by quinkblack, 16 months ago

I don't know about FFmpeg-Kit, especially why encoder has yuv420p pixel format while -hwaccel_output_format mediacodec.

Firstly, try if pix_fmt nv12 works: -c:v h264_mediacodec -ndk_codec 1 -pix_fmt nv12.

Secondly, try if -c:v h264_mediacodec -i test.mp4 -c:v h264_mediacodec -ndk_codec 1 -pix_fmt nv12 works, which means decoding to buffer and feed it to encoder, rather than decoding to surface.

I have some pending patches, but unlikely related to this ticket.

http://ffmpeg.org/pipermail/ffmpeg-devel/2022-December/304671.html
patch 2 3/7 has been replaced by patch v3 3/7.
http://ffmpeg.org/pipermail/ffmpeg-devel/2022-December/304717.html

in reply to:  3 comment:4 by Ajay Shrestha, 16 months ago

Replying to quinkblack:

I don't know about FFmpeg-Kit, especially why encoder has yuv420p pixel format while -hwaccel_output_format mediacodec.

Firstly, try if pix_fmt nv12 works: -c:v h264_mediacodec -ndk_codec 1 -pix_fmt nv12.

Secondly, try if -c:v h264_mediacodec -i test.mp4 -c:v h264_mediacodec -ndk_codec 1 -pix_fmt nv12 works, which means decoding to buffer and feed it to encoder, rather than decoding to surface.

I have some pending patches, but unlikely related to this ticket.

http://ffmpeg.org/pipermail/ffmpeg-devel/2022-December/304671.html
patch 2 3/7 has been replaced by patch v3 3/7.
http://ffmpeg.org/pipermail/ffmpeg-devel/2022-December/304717.html

1.I tried with -pxfmt nv12, and got the same gibberish result
2.Got the error Error while opening decoder for input stream when trying to decoding to buffer

I have attached the console log for both of them (console_output_nv12.log and console_output_decoding_to_buffer.log)

by Ajay Shrestha, 16 months ago

Attachment: console_output_nv12.log added

by Ajay Shrestha, 16 months ago

comment:5 by quinkblack, 16 months ago

The device is an emulator. I think there is no real hardware decoder/encoder on the emulator, only software implementation. FFmpeg mediacodec wrapper via JNI skip software decoders explicitly, so decoder doesn't work.

You can try the NDK implementation, put an '-ndk_codec 1' before -i

-ndk_codec 1 -c:v h264_mediacodec -i

I guess it will work. It's only for test. There's no sense to use MediaCodec software decoder, since FFmpeg's software decoder do better, that's

ffmpeg -i test.mp4 -c:v h264_mediacodec out.mp4

Should work. There is no real mediacodec hardware encoder neither. The software implementition has YUV420P support, so -pix_fmt nv12 is unnecessary.

in reply to:  5 comment:6 by Ajay Shrestha, 16 months ago

Replying to quinkblack:

The device is an emulator. I think there is no real hardware decoder/encoder on the emulator, only software implementation. FFmpeg mediacodec wrapper via JNI skip software decoders explicitly, so decoder doesn't work.

You can try the NDK implementation, put an '-ndk_codec 1' before -i

-ndk_codec 1 -c:v h264_mediacodec -i

I guess it will work. It's only for test. There's no sense to use MediaCodec software decoder, since FFmpeg's software decoder do better, that's

ffmpeg -i test.mp4 -c:v h264_mediacodec out.mp4

Should work. There is no real mediacodec hardware encoder neither. The software implementition has YUV420P support, so -pix_fmt nv12 is unnecessary.

Still getting the same Gibberish output with both options. Attached the log below

by Ajay Shrestha, 16 months ago

Attachment: console_log_ndk.log added

by Ajay Shrestha, 16 months ago

Attachment: console_log_2.log added

comment:7 by quinkblack, 16 months ago

I have tested with ffmpeg cmd on an "Pixel 4 API 30 x64" emulator and got no problem. Sorry I can't reproduce the issue.

FFmpeg cmd:

https://drive.google.com/file/d/1wuIvcdHVuwdVM-83qt0PG_5FA1GNxJhl/view?usp=sharing

comment:8 by quinkblack, 16 months ago

By the way, Android system log is very helpful when debug mediacodec.

comment:9 by Ajay Shrestha, 16 months ago

[​https://drive.google.com/file/d/1wuIvcdHVuwdVM-83qt0PG_5FA1GNxJhl/view?usp=sharing]
I couldn't open this file. It has some kind of binary string. Could you please share again ?

in reply to:  9 comment:10 by Balling, 16 months ago

Replying to Ajay Shrestha:

[​https://drive.google.com/file/d/1wuIvcdHVuwdVM-83qt0PG_5FA1GNxJhl/view?usp=sharing]
I couldn't open this file. It has some kind of binary string. Could you please share again ?

That is an executable.

comment:11 by quinkblack, 2 months ago

Resolution: worksforme
Status: newclosed
Note: See TracTickets for help on using tickets.