Opened 5 years ago

Closed 5 years ago

#1189 closed enhancement (fixed)

Seeking to timestamp 0 without AVSEEK_FLAG_BACKWARD results in seek to timestamp of 12

Reported by: mbradshaw Owned by:
Priority: minor Component: documentation
Version: git-master Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Trying to seek to a timestamp 0 without AVSEEK_FLAG_BACKWARD causes the stream to seek to a timestamp of 12. Adding the flag AVSEEK_FLAG_BACKWARD fixes this and results in seeking to the first frame.

Consequently, this makes the call avformat_seek_file(formatContext, 0, 0, 0, 0) seek to a timestamp of 12 instead of 0 as well.

If you run the program below, all the frames will be saved out in the correct order. However, if AVSEEK_FLAG_BACKWARD is removed, the first 11 frames are skipped (fyi the first frame is a repeat frame, so 12 with the repeat).

The attached video sample is an MOV container with with one mpeg2video stream. It was created from a source H264 sample with ffmpeg with the command:
ffmpeg -i TimeCode.mov -an -vcodec mpeg2video mpeg2.mov

I do not believe AVSEEK_FLAG_BACKWARD should be necessary here in this case. I believe this behavior shows up in other files as well (that is, seeking to a ts of 0 without AVSEEK_FLAG_BACKWARD ends up skipping the first keyframe and seeking to the second keyframe), but haven't confirmed this.

#include <fstream>
#include <iostream>

extern "C"
{
#include <avcodec.h>
#include <avformat.h>
#include <swscale.h>
};

void saveFrame(const AVFrame* frame, int width, int height, int frameNumber)
{
    char filename[32];
    sprintf(filename, "frame%d.ppm", frameNumber);
    std::ofstream file(filename, std::ios_base::binary | std::ios_base::trunc | std::ios_base::out);

    if (!file.good())
    {
        throw std::runtime_error("Unable to open the file to write the frame");
    }

    file << "P5\n" << width << '\n' << height << "\n255\n";

    for (int i = 0; i < height; ++i)
    {
        file.write((char*)(frame->data[0] + i * frame->linesize[0]), width);
    }
}

int main()
{
    av_register_all();
    AVFrame* frame = avcodec_alloc_frame();
    if (!frame)
    {
        return 1;
    }

    AVFormatContext* formatContext = NULL;
    if (avformat_open_input(&formatContext, "mpeg2.mov", NULL, NULL) != 0)
    {
        av_free(frame);
        return 1;
    }

    if (avformat_find_stream_info(formatContext, NULL) < 0)
    {
        av_free(frame);
        av_close_input_file(formatContext);
        return 1;
    }

    if (formatContext->nb_streams < 1 || formatContext->streams[0]->codec->codec_type != AVMEDIA_TYPE_VIDEO)
    {
        av_free(frame);
        av_close_input_file(formatContext);
        return 1;
    }

    AVStream* stream = formatContext->streams[0];
    AVCodecContext* codecContext = stream->codec;

    codecContext->codec = avcodec_find_decoder(codecContext->codec_id);
    if (codecContext->codec == NULL)
    {
        av_free(frame);
        avcodec_close(codecContext);
        av_close_input_file(formatContext);
        return 1;
    }
    else if (avcodec_open2(codecContext, codecContext->codec, NULL) != 0)
    {
        av_free(frame);
        avcodec_close(codecContext);
        av_close_input_file(formatContext);
        return 1;
    }

    avcodec_flush_buffers(codecContext);
    
    std::cout << "Seek successful? " << (av_seek_frame(formatContext, -1, 0, AVSEEK_FLAG_BACKWARD) >= 0) << std::endl;

    AVPacket packet;
    av_init_packet(&packet);

    std::ofstream stats("stats.txt");

    int frameNumber = 0;
    while (av_read_frame(formatContext, &packet) == 0)
    {
        std::cout << "key packet? " << (packet.flags & AV_PKT_FLAG_KEY) << std::endl;
        if (packet.stream_index == stream->index)
        {
            int frameFinished = 0;
            avcodec_decode_video2(codecContext, frame, &frameFinished, &packet);

            if (frameFinished)
            {
                saveFrame(frame, codecContext->width, codecContext->height, frameNumber++);
                stats << "repeat: " << frame->repeat_pict << "\tkeyframe: " << frame->key_frame << "\tbest_ts: " << frame->best_effort_timestamp << '\n';
            }
        }
    }
    
    av_free_packet(&packet);

    if (codecContext->codec->capabilities & CODEC_CAP_DELAY)
    {
        av_init_packet(&packet);
        int frameFinished = 0;
        int result = 0;
        while ((result = avcodec_decode_video2(codecContext, frame, &frameFinished, &packet)) >= 0 && frameFinished)
        {
            if (frameFinished)
            {
                saveFrame(frame, codecContext->width, codecContext->height, frameNumber++);
                stats << "repeat: " << frame->repeat_pict << "\tkeyframe: " << frame->key_frame << "\tbest_ts: " << frame->best_effort_timestamp << '\n';
            }
        }
    }

    av_free(frame);
    avcodec_close(codecContext);
    av_close_input_file(formatContext);
}

Attachments (1)

mpeg2.mov (1.1 MB) - added by mbradshaw 5 years ago.

Download all attachments as: .zip

Change History (5)

Changed 5 years ago by mbradshaw

comment:1 Changed 5 years ago by reimar

  • Resolution set to invalid
  • Status changed from new to closed

As ffprobe -show_packets shows, the first frame has a (dts) time stamp of -1.
Thus seeking to the first frame when seeking to timestamp 0 would be wrong.
As far as I can tell FFmpeg behaves exactly as it should, and doesn't really have much choice in the matter.

Note: I admit that it is up to discussion whether FFmpeg should rather be seeking by dts or pts, though by dts gives probably more logical/reliable results.
Whether it should avoid creating files with negative dts yet another. I can't really comment much on either.
It also looks like FFmpeg sets the start time to 0 and not -1, which while I believe is correct is rather confusing with that dts/pts mess.
But even without that there are many, many file formats that almost never start with a time stamp of 0.

Still, overall I do not believe there is any bug, even if the behaviour is maybe needlessly confusing or too badly documented, so there might be something for which you might want to open an enhancement request (or change this one to such if possible).

comment:2 Changed 5 years ago by mbradshaw

  • Component changed from undetermined to documentation
  • Resolution invalid deleted
  • Status changed from closed to reopened
  • Type changed from defect to enhancement

Thanks Reimar. I'll change this to an enhancement request and reopen it as a request to update the relevant documentation. I thought it was using pts, and since the start time is zero I couldn't see why it was acting the way it was. If dts is used though, the behavior makes more sense.

I can't comment on seeking by pts vs dts. Personally, I would like it if negative dts could be avoided, but I can't comment much on this either.

comment:3 Changed 5 years ago by cehoyos

  • Priority changed from normal to minor

Is this still a problem or was the documentation improved accordingly?

comment:4 Changed 5 years ago by mbradshaw

  • Resolution set to fixed
  • Status changed from reopened to closed

I was thinking it would be nice to put a comment in the docs for av_seek_frame and avformat_seek_file informing the user they may want to check AVInputFormat.flags when seeking (for things like AVFMT_NO_BYTE_SEEK or AVFMT_SEEK_TO_PTS), but I suppose that could be opened as a different issue, as it deals with more than just this PTS vs DTS ticket.

Most relevant demuxers haven't been updated with the new AVFMT_SEEK_TO_PTS flag, but I don't know which ones to update and I don't have time to dig through all the code to find which ones seek by PTS, so I think those can be opened as new tickets as they're found.

I'll close this as fixed by Michael Niedermayer, as I believe his patch appropriately fixes this, and now the remaining work is just to make sure it's used properly.

Note: See TracTickets for help on using tickets.