Opened 6 years ago

Closed 6 years ago

#5593 closed defect (fixed)

OS X h264_videotoolbox encoder and codecpar API race condition with extradata when using MP4 / mov

Reported by: vade Owned by: Rick Kern
Priority: normal Component: avcodec
Version: git-master Keywords: h264_videotoolbox, extradata, codecpar
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no


Summary of the bug:

On OS X, using the new send frame / receive packet / codec parameter API, it appears that there is a race condition or a lack of setting of the extra data / extra data size when specifying h264_videotoolbox encoder.

We create do our normal output context setup, make a stream, and using the new codecparam API we:

  • avcodec_alloc_context3 a new AVCodecContext and set it up as per our streams setting.
  • avcodec_open2 the codec
  • avcodec_parameters_from_context to our target stream

We then avcodec_send_frame / avcodec_receive_packet as expected, with no errors, and mux our resulting packets using av_interleaved_write_frame.

Our resulting MP4 has invalid video samples as per Apples Quicktime Player, resulting in errors in the console :

5/27/16 12:40:50.282 PM VTDecoderXPCService[26681]: [12:40:50.282] H264VideoDecoder_DecodeFrame signalled err=-12910 (err) (createJVTLibInstance failed) at /Library/Caches/ line 2477

Quicktime Player 7, Quicklook, and other video programs also have errors opening the file.

if we switch our encoder to libx264, and everything works as expected with no errors in sight.

One can fix the above issue if on the first output packet received from avcodec_receive_packet, we set our streams codecpar extra data / extra data size from our codec context above.

This is the race condition. Pseudocode :

 ret = avcodec_send_frame(mOutputVideoEncoderContext, inputFrame);
            if( ret >= 0 ) {
                ret = avcodec_receive_packet(mOutputVideoEncoderContext, outputPacket);
                if( ret >= 0 ) {
                    encodedSuccess = 1;
                    outputPacket->stream_index = videoStream->index;
                    outputPacket->duration = mAvgVideoPacketDuration;
                    // DEBUG FOR VIDEO TOOLBOX BUG
                    if( videoStream->codecpar->extradata == NULL ) {
                        videoStream->codecpar->extradata = mOutputVideoEncoderContext->extradata;
                        videoStream->codecpar->extradata_size = mOutputVideoEncoderContext->extradata_size;
                else {
                    std::cout << "avcodec_receive_packet got " << av_err2str(ret) << std::endl;
            else {
                std::cout << "avcodec_send_frame got " << av_err2str(ret) << std::endl;

This bug was verified on IRC by rkern, and I was requested to make a trac issue for official reporting of the bug.

Please note this issue is relevant to git master as of commit d93495c95411b95cfc5c050dbabfcdff1c10890a

Thank you!

Change History (5)

comment:1 by vade, 6 years ago

Summary: OS X h264_videotoolbox encoder and codecpar API race condition with extradite when using MP4 / movOS X h264_videotoolbox encoder and codecpar API race condition with extradata when using MP4 / mov

comment:2 by Hendrik, 6 years ago

This would appear to be a bug in the videotoolbox encoder, and unrelated to the new encode API.
Encoders need to set the extradata during init to work properly with formats that require global metadata. libx264 does this.

comment:3 by vade, 6 years ago

Thanks ! Do i need to edit the bug to correctly tag video toolbox encoder as the source of the issue?

comment:4 by Rick Kern, 6 years ago

Owner: set to Rick Kern
Status: newopen

comment:5 by Rick Kern, 6 years ago

Resolution: fixed
Status: openclosed
Note: See TracTickets for help on using tickets.