[Android] MediaCodec h264 decoder fails on 10-bit input ('This is a bug, please report it' in output)

[bubbleguuum]

Using custom Android FFmpeg 6.1 build on Pixel 4a.

Decoding a 10-bit h264 file (yuv420p10le) with MediaCodec fails with a "This is a bug, please report it" trace, hence this report.

I suppose it fails because 10-bit is not supported by h264_mediacodec.
Ideally it should bail out immediately on this error but it does not.

There is also a "FORTIFY: pthread_mutex_lock called on a destroyed mutex" warning at the end.

Input sample file can be download here:
​https://www.dropbox.com/sh/6iy4gxgsfn14opq/AAAK_L1M_NwZwBPb5IkEPh7Ga?dl=0

$ ./ffmpeg -hide_banner -hwaccel mediacodec -i /sdcard/Movies/\[Underwater]\ Another\ -\ sample\ H264\ Hi10P\ 720p.avi  -f null -                                            
Input #0, avi, from '/sdcard/Movies/[Underwater] Another - sample H264 Hi10P 720p.avi':
  Duration: 09:10:31.48, start: 0.000000, bitrate: 4 kb/s
  Stream #0:0: Video: h264 (High 10) (AVC1 / 0x31435641), yuv420p10le(tv, bt709, progressive), 1280x720, 1442 kb/s, 23.98 fps, 23.98 tbr, 23.98 tbn
  Stream #0:1: Audio: aac (LC) ([255][0][0][0] / 0x00FF), 44100 Hz, stereo, fltp, 128 kb/s
[h264_mediacodec @ 0xb4000070ac2d6510] Both surface and native_window are NULL
[h264_mediacodec @ 0xb4000070ac2d6510] Using surface 0x0
[h264_mediacodec @ 0xb4000070ac2d6510] No Java virtual machine has been registered
[h264_mediacodec @ 0xb4000070ac2d6510] Failed to getCodecNameByType
[h264_mediacodec @ 0xb4000070ac2d6510] Output crop parameters top=0 bottom=719 left=0 right=1279, resulting dimensions width=1280 height=720
[h264_mediacodec @ 0xb4000070ac2d6510] MediaCodec started successfully: codec = c2.qti.avc.decoder, ret = 0
[h264_mediacodec @ 0xb4000070ac2d6510] No Java virtual machine has been registered
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_mediacodec) -> wrapped_avframe (native))
  Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
[h264_mediacodec @ 0xb4000070ac2d6510] Input packet is missing PTS
    Last message repeated 1 times
[h264_mediacodec @ 0xb4000070ac2d6510] Failed to queue input buffer (status = -10000)
[vist#0:0/h264 @ 0xb40000706c2d8690] Error submitting packet to decoder: Generic error in an external library
[h264_mediacodec @ 0xb4000070ac2d6510] Input packet is missing PTS
[h264_mediacodec @ 0xb4000070ac2d6510] Failed to queue input buffer (status = -10000)
[vist#0:0/h264 @ 0xb40000706c2d8690] Error submitting packet to decoder: Generic error in an external library
[vist#0:0/h264 @ 0xb40000706c2d8690] A decoder returned an unexpected error code. This is a bug, please report it.
[vist#0:0/h264 @ 0xb40000706c2d8690] Error processing packet in decoder: Internal bug, should not have happened
[vist#0:0/h264 @ 0xb40000706c2d8690] Decoder thread returned error: Internal bug, should not have happened
[vost#0:0/wrapped_avframe @ 0xb40000709c2d82d0] No filtered frames for output stream, trying to initialize anyway.
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf60.16.100
  Stream #0:0: Video: wrapped_avframe, nv12(progressive), 1280x720, q=2-31, 200 kb/s, 23.98 fps, 23.98 tbn
    Metadata:
      encoder         : Lavc60.31.102 wrapped_avframe
  Stream #0:1: Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s
    Metadata:
      encoder         : Lavc60.31.102 pcm_s16le
[vist#0:0/h264 @ 0xb40000706c2d8690] Decode error rate 1 exceeds maximum 0.666667x    
[out#0/null @ 0xb40000704c2de190] video:0kB audio:15316kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
frame=    0 fps=0.0 q=0.0 Lsize=N/A time=00:01:28.88 bitrate=N/A speed=92.6x    
Conversion failed!
FORTIFY: pthread_mutex_lock called on a destroyed mutex (0xb40000701c2ddec8)
FORTIFY: pthread_mutex_lock called on a destroyed mutex (0xb40000701c2ddfe8)

Input file can be obtained here:

[quinkblack]

1. pthread_mutex_lock bug has been fixed on master branch.
2. The misleading error messages mostly come from error handing in ffmpeg cli. It can be silented by

./ffmpeg -xerror -hwaccel mediacodec -i ...

libavcodec/mediacodec already returned error code, I'm not sure whether anything need to be improved inside libavcodec/mediacodec regarding this issue.

3. There are other bugs in ffmpeg cli with master branch when testing this bug, see

https://ffmpeg.org/pipermail/ffmpeg-devel/2023-December/318493.html

[bubbleguuum]

In that scenario (inability to transcode the video track), I would expect the ffmpeg process to exit immediately instead of producing a file with just the audio track:

[out#0/null @ 0xb40000704c2de190] video:0kB audio:15316kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

Unless there is a command-line option that I'm not aware of to have this behavior (exiting immediately) ?

The thing is that I have a script that invoke ffmpeg with some command-line options and if it fails has a fallback using different options. This does not work if ffmpeg is not exiting on inability to video transcode as shown in this example.

[quinkblack]

Replying to bubbleguuum:

In that scenario (inability to transcode the video track), I would expect the ffmpeg process to exit immediately instead of producing a file with just the audio track:

[out#0/null @ 0xb40000704c2de190] video:0kB audio:15316kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

Unless there is a command-line option that I'm not aware of to have this behavior (exiting immediately) ?

As in comment:1, -xerror

[bubbleguuum]

With -xerror, it silences some of the error output (the 'this is a bug, please report' paragraph), but the encoding proceeds (just encoding audio) instead of exiting early because of the inability to encode video.
That's of course without your 2 patches posted on the mailing list, that I did not try and that may fix this issue:

69|sunfish:/data/local/tmp $ ./ffmpeg -xerror -hide_banner -hwaccel mediacodec -i /sdcard/Movies/\[Underwater]\ Another\ -\ sample\ H264\ Hi10P\ 720p.avi  -f null -                                    
Input #0, avi, from '/sdcard/Movies/[Underwater] Another - sample H264 Hi10P 720p.avi':
  Duration: 09:10:31.48, start: 0.000000, bitrate: 4 kb/s
  Stream #0:0: Video: h264 (High 10) (AVC1 / 0x31435641), yuv420p10le(tv, bt709, progressive), 1280x720, 1442 kb/s, 23.98 fps, 23.98 tbr, 23.98 tbn
  Stream #0:1: Audio: aac (LC) ([255][0][0][0] / 0x00FF), 44100 Hz, stereo, fltp, 128 kb/s
[h264_mediacodec @ 0xb400007a36961150] Both surface and native_window are NULL
[h264_mediacodec @ 0xb400007a36961150] Using surface 0x0
[h264_mediacodec @ 0xb400007a36961150] No Java virtual machine has been registered
[h264_mediacodec @ 0xb400007a36961150] Failed to getCodecNameByType
[h264_mediacodec @ 0xb400007a36961150] Output crop parameters top=0 bottom=719 left=0 right=1279, resulting dimensions width=1280 height=720
[h264_mediacodec @ 0xb400007a36961150] MediaCodec started successfully: codec = c2.qti.avc.decoder, ret = 0
[h264_mediacodec @ 0xb400007a36961150] No Java virtual machine has been registered
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_mediacodec) -> wrapped_avframe (native))
  Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
[h264_mediacodec @ 0xb400007a36961150] Input packet is missing PTS
    Last message repeated 1 times
[h264_mediacodec @ 0xb400007a36961150] Failed to queue input buffer (status = -10000)
[vist#0:0/h264 @ 0xb4000079f695e5b0] Error submitting packet to decoder: Generic error in an external library
[vist#0:0/h264 @ 0xb4000079f695e5b0] Error processing packet in decoder: Generic error in an external library
[vist#0:0/h264 @ 0xb4000079f695e5b0] Decoder thread returned error: Generic error in an external library
[vost#0:0/wrapped_avframe @ 0xb400007a26961e30] No filtered frames for output stream, trying to initialize anyway.
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf60.16.100
  Stream #0:0: Video: wrapped_avframe, nv12(progressive), 1280x720, q=2-31, 200 kb/s, 23.98 fps, 23.98 tbn
    Metadata:
      encoder         : Lavc60.31.102 wrapped_avframe
  Stream #0:1: Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s
    Metadata:
      encoder         : Lavc60.31.102 pcm_s16le
[vist#0:0/h264 @ 0xb4000079f695e5b0] Decode error rate 1 exceeds maximum 0.666667x    
[out#0/null @ 0xb4000079d695beb0] video:0kB audio:15316kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
frame=    0 fps=0.0 q=0.0 Lsize=N/A time=00:01:28.88 bitrate=N/A speed= 100x    
Conversion failed!
FORTIFY: pthread_mutex_lock called on a destroyed mutex (0xb4000079a69699b8)

[quinkblack]

The issue has been fixed on master branch, although I don't know which commit.

[quinkblack]

The issue has been fixed on master branch.