Last 11 Frames of H264 MOV Are Skipped When Decoding

[mbradshaw]

While working on a video reader I found I wasn't seeing all the frames from the video. It's possible I'm doing something wrong, but I haven't found any documentation that says I need to do anything special for decoding H264 video. The av_read_frame(formatContext, &packet) returns nonzero 11 frames before the end of the video.

I transcoded the video with ffmpeg to mpeg2video, and the transcoded video did not show the same problem.

Minimal Reproducible Code Sample

#include <fstream>

extern "C"
{
#include <avcodec.h>
#include <avformat.h>
#include <swscale.h>
};

void saveFrame(const AVFrame* frame, int width, int height, int frameNumber)
{
    char filename[32];
    sprintf(filename, "frame%d.ppm", frameNumber);
    std::ofstream file(filename, std::ios_base::binary |
                                 std::ios_base::trunc |
                                 std::ios_base::out);

    if (!file.good())
    {
        throw std::runtime_error("Unable to open the file to write the frame");
    }

    file << "P5\n" << width << '\n' << height << "\n255\n";

    for (int i = 0; i < height; ++i)
    {
        file.write((char*)(frame->data[0] + i * frame->linesize[0]), width);
    }
}

int main()
{
    av_register_all();
    AVFrame* frame = avcodec_alloc_frame();
    if (!frame)
    {
        return 1;
    }

    AVFormatContext* formatContext = NULL;
    if (avformat_open_input(&formatContext, "in.mov", NULL, NULL) != 0)
    {
        av_free(frame);
        return 1;
    }

    if (avformat_find_stream_info(formatContext, NULL) < 0)
    {
        av_free(frame);
        av_close_input_file(formatContext);
        return 1;
    }

    if (formatContext->nb_streams < 1 ||
        formatContext->streams[0]->codec->codec_type != AVMEDIA_TYPE_VIDEO)
    {
        av_free(frame);
        av_close_input_file(formatContext);
        return 1;
    }

    AVStream* stream = formatContext->streams[0];
    AVCodecContext* codecContext = stream->codec;

    codecContext->codec = avcodec_find_decoder(codecContext->codec_id);
    if (codecContext->codec == NULL)
    {
        av_free(frame);
        avcodec_close(codecContext);
        av_close_input_file(formatContext);
        return 1;
    }
    else if (avcodec_open2(codecContext, codecContext->codec, NULL) != 0)
    {
        av_free(frame);
        avcodec_close(codecContext);
        av_close_input_file(formatContext);
        return 1;
    }

    AVPacket packet;
    av_init_packet(&packet);

    std::ofstream stats("stats.txt");

    int frameNumber = 0;
    while (av_read_frame(formatContext, &packet) == 0)
    {
        if (packet.stream_index == stream->index)
        {
            int frameFinished = 0;
            avcodec_decode_video2(codecContext, frame, &frameFinished, &packet);

            if (frameFinished)
            {
                saveFrame(frame, codecContext->width, codecContext->height, frameNumber++);
                stats << "repeat: " << frame->repeat_pict
                      << "\tkeyframe: " << frame->key_frame
                      << "\tbest_ts: " << frame->best_effort_timestamp << '\n';
            }
        }
    }

    av_free_packet(&packet);
    av_free(frame);
    avcodec_close(codecContext);
    av_close_input_file(formatContext);
}

Config options: ./configure --arch=x86 --enable-shared --disable-static
FFmpeg/libavcodec compiler: GCC/MinGW
Compiler for above sample: VC++ 2010
Expected output: frame0.ppm to frame 360.ppm
Actual output: frame0.ppm to frame 349.ppm

ffmpeg.exe -i in.mov

ffmpeg version 0.10.2.git-a45605a Copyright (c) 2000-2012 the FFmpeg developers
  built on Apr  4 2012 12:58:46 with gcc 4.6.1
  configuration: --arch=x86 --enable-shared --disable-static
  libavutil      51. 44.100 / 51. 44.100
  libavcodec     54. 12.100 / 54. 12.100
  libavformat    54.  3.100 / 54.  3.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 66.101 /  2. 66.101
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 10.100 /  0. 10.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'in.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 512
    compatible_brands: qt
    encoder         : Lavf54.3.100
  Duration: 00:00:15.13, start: 0.000000, bitrate: 1002 kb/s
    Stream #0:0(eng): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p, 852x480 [SAR 1:1 DAR 71:40], 1000 kb/s, 23.92 fps, 24 tbr, 600 tbn, 47.95 tbc
    Metadata:
      handler_name    : ♂DataHandler

Sample video attached.

[mbradshaw]

Problematic video file

[Carl Eugen Hoyos]

The sample contains 362 frames ("00:00:00:00" is repeated once and the last frame is "00:00:15:00"), ffmpeg decodes all of them (bitexact if compared with the reference decoder).

[reimar]

Please read the documentation.
For example the one for avcodec_decode_video2 says:

• @note Codecs which have the CODEC_CAP_DELAY capability set have a delay
• between input and output, these need to be fed with avpkt->data=NULL,
• avpkt->size=0 at the end to return the remaining frames.

Since you are not doing that you do not have the chance of getting all frames decoded with most codecs.

[mbradshaw]

Thank you, that was what I was overlooking. Sorry for the false alarm.