Opened 14 months ago

Last modified 14 months ago

#8865 new defect

Lost frames when decode by mediacodec

Reported by: jiyang Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: mediacodec
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

All code most sections same with doc/examples/hw_decode.c, I add mediacodec section, but when decode, I found always lost some frames.

How to reproduce:

    AVMediaCodecContext *mediacodec_ctx = av_mediacodec_alloc_context();
    av_mediacodec_default_init(decoder_ctx, mediacodec_ctx, surface);

    int eof = 0;
    while (1) {
        if (!eof) {
            ret = av_read_frame(input_ctx, &packet);
            LOGD("======= Read frame to packet: is_video: %d. %s =======",
                 packet.stream_index == video_stream,
                 av_err2str(ret));

            if (video_stream == packet.stream_index) {
                if (ret == AVERROR_EOF) {
                    eof = 1;
                    packet.data = NULL;
                    packet.size = 0;
                }
                double rq = av_q2d(video->time_base);
                LOGD("Packet: pts: %f, dts: %f", packet.pts * rq, packet.dts * rq);
                ret = avcodec_send_packet(decoder_ctx, &packet);
                LOGD("Send packet to decoder. %s", av_err2str(ret));
                if (ret == AVERROR(EAGAIN)) {
                    //
                } else if (ret < 0) {
                    break;
                }
            }
        }

        if (packet.stream_index == video_stream) {
            ret = decode_write(decoder_ctx, surface, frame, sw_frame, video);
            if (ret < 0) {
                break;
            }
        }

        av_packet_unref(&packet);
    }


static int decode_write(AVCodecContext *avctx, void *surface, AVFrame *frame, AVFrame *sw_frame,
                        AVStream *video_stream) {
    int ret = 0;

    while (1) {
        ret = avcodec_receive_frame(avctx, frame);
        LOGD("========= Receive frame from codec [%s] =========", av_err2str(ret));
        if (ret == AVERROR(EAGAIN)) {
            return 0;
        } else if (ret == AVERROR_EOF) {
            return ret;
        } else if (ret < 0) {
            return ret;
        }

        enum AVPixelFormat pixelFormat = (enum AVPixelFormat) (frame->format);
        double time_sec = frame->pts * av_q2d(video_stream->time_base);
        LOGD("Frame: %d, pts: %f, format: %s",
             avctx->frame_number,
             time_sec,
             av_get_pix_fmt_name(pixelFormat));

        int64_t time_nsec = time_sec * 1000000; // nanoseconds
        AVMediaCodecBuffer *mediaCodecBuffer = (AVMediaCodecBuffer *) frame->data[3];
        if (mediaCodecBuffer) {
            ret = av_mediacodec_render_buffer_at_time(mediaCodecBuffer, time_nsec);
            LOGD("Render buffer at %lld. %s", time_nsec, av_err2str(ret));
            av_frame_unref(frame);
        }
    }

    return ret;
}

Log:

D/HwDecode: Frame: 1, pts: 0.000000, format: mediacodec
D/HwDecode: Frame: 2, pts: 0.040000, format: mediacodec
D/HwDecode: Frame: 3, pts: 0.080000, format: mediacodec
D/HwDecode: Frame: 4, pts: 0.160000, format: mediacodec
D/HwDecode: Frame: 5, pts: 0.440000, format: mediacodec
D/HwDecode: Frame: 6, pts: 0.920000, format: mediacodec
D/HwDecode: Frame: 7, pts: 1.000000, format: mediacodec
D/HwDecode: Frame: 8, pts: 1.040000, format: mediacodec
D/HwDecode: Frame: 9, pts: 1.080000, format: mediacodec
D/HwDecode: Frame: 10, pts: 1.120000, format: mediacodec
D/HwDecode: Frame: 11, pts: 1.160000, format: mediacodec
D/HwDecode: Frame: 12, pts: 1.200000, format: mediacodec
D/HwDecode: Frame: 13, pts: 1.240000, format: mediacodec
D/HwDecode: Frame: 14, pts: 1.280000, format: mediacodec
D/HwDecode: Frame: 15, pts: 1.480000, format: mediacodec
D/HwDecode: Frame: 16, pts: 1.520000, format: mediacodec
D/HwDecode: Frame: 17, pts: 1.600000, format: mediacodec
D/HwDecode: Frame: 18, pts: 1.640000, format: mediacodec
D/HwDecode: Frame: 19, pts: 1.680000, format: mediacodec
D/HwDecode: Frame: 20, pts: 1.720000, format: mediacodec
D/HwDecode: Frame: 21, pts: 1.760000, format: mediacodec
D/HwDecode: Frame: 22, pts: 1.800000, format: mediacodec
D/HwDecode: Frame: 23, pts: 1.840000, format: mediacodec
D/HwDecode: Frame: 24, pts: 1.880000, format: mediacodec
D/HwDecode: Frame: 25, pts: 1.920000, format: mediacodec

Lost frames: [0.12, 0.22~0.4, 0.48~0.86, 0.96], and lost is random.

Change History (3)

comment:1 by jiyang, 14 months ago

packets return by av_read_frame is normal:

D: Packet: pts: 0.000000, dts: -0.040000
D: Packet: pts: 0.080000, dts: 0.000000
D: Packet: pts: 0.040000, dts: 0.040000
D: Packet: pts: 0.160000, dts: 0.080000
D: Packet: pts: 0.120000, dts: 0.120000
D: Packet: pts: 0.240000, dts: 0.160000
D: Packet: pts: 0.200000, dts: 0.200000
D: Packet: pts: 0.320000, dts: 0.240000
D: Packet: pts: 0.280000, dts: 0.280000
D: Packet: pts: 0.400000, dts: 0.320000
D: Packet: pts: 0.360000, dts: 0.360000
D: Packet: pts: 0.480000, dts: 0.400000
D: Packet: pts: 0.440000, dts: 0.440000
D: Packet: pts: 0.560000, dts: 0.480000
D: Packet: pts: 0.520000, dts: 0.520000
D: Packet: pts: 0.640000, dts: 0.560000
D: Packet: pts: 0.600000, dts: 0.600000
D: Packet: pts: 0.720000, dts: 0.640000
D: Packet: pts: 0.680000, dts: 0.680000
D: Packet: pts: 0.800000, dts: 0.720000
D: Packet: pts: 0.760000, dts: 0.760000
D: Packet: pts: 0.880000, dts: 0.800000
D: Packet: pts: 0.840000, dts: 0.840000
D: Packet: pts: 0.960000, dts: 0.880000
D: Packet: pts: 0.920000, dts: 0.920000
D: Packet: pts: 1.040000, dts: 0.960000
D: Packet: pts: 1.000000, dts: 1.000000
D: Packet: pts: 1.120000, dts: 1.040000
D: Packet: pts: 1.080000, dts: 1.080000
D: Packet: pts: 1.200000, dts: 1.120000
D: Packet: pts: 1.160000, dts: 1.160000
D: Packet: pts: 1.280000, dts: 1.200000
D: Packet: pts: 1.240000, dts: 1.240000
D: Packet: pts: 1.360000, dts: 1.280000
D: Packet: pts: 1.320000, dts: 1.320000
D: Packet: pts: 1.440000, dts: 1.360000
D: Packet: pts: 1.400000, dts: 1.400000
D: Packet: pts: 1.520000, dts: 1.440000
D: Packet: pts: 1.480000, dts: 1.480000
D: Packet: pts: 1.600000, dts: 1.520000
D: Packet: pts: 1.560000, dts: 1.560000
D: Packet: pts: 1.680000, dts: 1.600000
D: Packet: pts: 1.640000, dts: 1.640000
D: Packet: pts: 1.760000, dts: 1.680000
D: Packet: pts: 1.720000, dts: 1.720000
D: Packet: pts: 1.840000, dts: 1.760000
D: Packet: pts: 1.800000, dts: 1.800000
D: Packet: pts: 1.920000, dts: 1.840000
D: Packet: pts: 1.880000, dts: 1.880000
D: Packet: pts: 1.960000, dts: 1.920000

comment:2 by Carl Eugen Hoyos, 14 months ago

Priority: importantnormal

Is this issue not reproducible with ffmpeg, the command line utility?

comment:3 by jiyang, 14 months ago

I don't know how to use mediacodec with ffmpeg command line interface in Android Phone.

Note: See TracTickets for help on using tickets.