Opened 3 years ago

Last modified 2 years ago

#2338 new defect

avformat_seek_file seeking to wrong frame for AVCHD sample

Reported by: rmk Owned by:
Priority: normal Component: avformat
Version: unspecified Keywords: h264 seek
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

I have been observing that for files from a certain camera avformat_seek_file did not seek to the closest keyframe as requested (it seeked past the requested interval, although suitable frames do exist in the specified interval).

To reproduce this using ffmpeg, please try the file from ticket 2336 and attempt to seek to position 0.96 using -ss 0.96 as an input option. Looking at the code in ffmpeg_opt.c ffmpeg issues a key_frame seek for the interval (INT64_MIN, 0.96 in timebase units, 0.96 in timebase units), so the first frame decoded by ffmpeg should be the keyframe closest to 0.96 _before_ 0.96 but the result is different.

From debug output added to h264_parser.c I get the following frame numbers (calculated by pts) for key frames (the fact that the decoder doesn't see those as keyframes but the parser does, is a separate issue I described in http://ffmpeg.org/pipermail/libav-user/2013-March/003915.html but I am not sure it should be filed as a ticket, probably yes).

PTS Sec. Frame number
67515 0 0
89115 0.24 6
99915 0.36 9
110715 0.48 12
121515 0.6 15
132315 0.72 18
143115 0.84 21
153915 0.96 24
164715 1.08 27
175515 1.2 30
186315 1.32 33

So 0.96 should work and be exact but from looking at the file, it appears it is really returning the next keyframe (27 at 1.08 Seconds).

Command lines to reproduce:

ffmpeg -ss 0.96 -i ~/tmp/gh2-50i-psf.mts -an -sn -vframes 1 ~/tmp/out096.jpg

Produces frame #27 at 1.08 seconds

ffmpeg -ss 0.92 -i ~/tmp/gh2-50i-psf.mts -an -sn -vframes 1 ~/tmp/out092.jpg

Produces frame #24 at 0.96 seconds.

Looking at the code an API doc I do not see any other explanation than avformat_seek_file not seeking to the right keyframe (it appears it is one too late in this case and others I have observed).

I attached the patch with the log output for the h264 parser. I am not sure I am interpreting this correctly but from what I know about the camera's GOP size, it does make sense.

Here's the output of the runs.

ffmpeg -ss 0.96 -i ~/tmp/gh2-50i-psf.mts -an -sn -vframes 1 -y ~/tmp/out096.jpg
ffmpeg version N-50562-g199db97 Copyright (c) 2000-2013 the FFmpeg developers

built on Mar 8 2013 10:08:49 with llvm-gcc 4.2.1 (LLVM build 2336.11.00)
configuration:
libavutil 52. 18.100 / 52. 18.100
libavcodec 54. 92.100 / 54. 92.100
libavformat 54. 63.104 / 54. 63.104
libavdevice 54. 3.103 / 54. 3.103
libavfilter 3. 42.103 / 3. 42.103
libswscale 2. 2.100 / 2. 2.100
libswresample 0. 17.102 / 0. 17.102

Input #0, mpegts, from '/Users/krueger/tmp/gh2-50i-psf.mts':

Duration: 00:00:01.38, start: 0.710178, bitrate: 46737 kb/s
Program 1

Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn, 50 tbc
Stream #0:1[0x1100]: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 192 kb/s
Stream #0:2[0x1200]: Subtitle: hdmv_pgs_subtitle ([144][0][0][0] / 0x0090)

Output #0, image2, to '/Users/krueger/tmp/out096.jpg':

Metadata:

encoder : Lavf54.63.104
Stream #0:0: Video: mjpeg, yuvj420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 90k tbn, 25 tbc

Stream mapping:

Stream #0:0 -> #0:0 (h264 -> mjpeg)

Press [q] to stop, ? for help
[h264 @ 0x7f9c42bc6000] Missing reference picture, default is 0
[h264 @ 0x7f9c42bc6000] decode_slice_header error
[h264 @ 0x7f9c42bc6c00] Missing reference picture, default is 0
[h264 @ 0x7f9c42bc6c00] decode_slice_header error
frame= 1 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
video:140kB audio:0kB subtitle:0 global headers:0kB muxing overhead -100.015333%

ffmpeg -ss 0.90 -i ~/tmp/gh2-50i-psf.mts -an -sn -vframes 1 -y ~/tmp/out090.jpg
ffmpeg version N-50562-g199db97 Copyright (c) 2000-2013 the FFmpeg developers

built on Mar 8 2013 10:08:49 with llvm-gcc 4.2.1 (LLVM build 2336.11.00)
configuration:
libavutil 52. 18.100 / 52. 18.100
libavcodec 54. 92.100 / 54. 92.100
libavformat 54. 63.104 / 54. 63.104
libavdevice 54. 3.103 / 54. 3.103
libavfilter 3. 42.103 / 3. 42.103
libswscale 2. 2.100 / 2. 2.100
libswresample 0. 17.102 / 0. 17.102

Input #0, mpegts, from '/Users/krueger/tmp/gh2-50i-psf.mts':

Duration: 00:00:01.38, start: 0.710178, bitrate: 46737 kb/s
Program 1

Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn, 50 tbc
Stream #0:1[0x1100]: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 192 kb/s
Stream #0:2[0x1200]: Subtitle: hdmv_pgs_subtitle ([144][0][0][0] / 0x0090)

Output #0, image2, to '/Users/krueger/tmp/out090.jpg':

Metadata:

encoder : Lavf54.63.104
Stream #0:0: Video: mjpeg, yuvj420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 90k tbn, 25 tbc

Stream mapping:

Stream #0:0 -> #0:0 (h264 -> mjpeg)

Press [q] to stop, ? for help
frame= 1 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
video:141kB audio:0kB subtitle:0 global headers:0kB muxing overhead -100.015187%


Attachments (1)

h264_parser_debug.patch (1.2 KB) - added by rmk 3 years ago.

Download all attachments as: .zip

Change History (10)

Changed 3 years ago by rmk

comment:1 Changed 3 years ago by rmk

One more thing I just noticed is that the stream start time of the video (and thus the start time of the container) seems to be wrong. It is detected as 63916 (0,7101777778 as reported by ffmpeg on the command line). However, the pts of the first decoded frame is 67515. Do you want me to make a new report for this? The wrong start time would certainly explain seeking inaccuracy to a certain extend as it is used in open_input_file to compute the seek target.

comment:2 Changed 3 years ago by rmk

The place in the code, where the packet PTS is "corrected" to the strange value 63916, is the following in compute_pkt_fields

if (pc && pc->dts_sync_point >= 0) {

we have synchronization info from the parser
int64_t den = st->codec->time_base.den * (int64_t) st->time_base.num;
if (den > 0) {

int64_t num = st->codec->time_base.num * (int64_t) st->time_base.den;
if (pkt->dts != AV_NOPTS_VALUE) {

got DTS from the stream, update reference timestamp
st->reference_dts = pkt->dts - pc->dts_ref_dts_delta * num / den;
pkt->pts = pkt->dts + pc->pts_dts_delta * num / den;

here the correct pts 67515 is replaced by DTS (63915) + 1, i.e. 63916

} else if (st->reference_dts != AV_NOPTS_VALUE) {

compute DTS based on reference timestamp
pkt->dts = st->reference_dts + pc->dts_ref_dts_delta * num / den;
pkt->pts = pkt->dts + pc->pts_dts_delta * num / den;

}
if (pc->dts_sync_point > 0)

st->reference_dts = pkt->dts; new reference

}

}

comment:3 Changed 3 years ago by cehoyos

Is this still reproducible?
Does it behave differently with -flags2 showall ?

Last edited 3 years ago by cehoyos (previous) (diff)

comment:4 Changed 3 years ago by rmk

Where are samples stored that are uploaded via FTP (in this case the one for #2336)? I want to make sure I am using the same sample to check and I do not have it on my local disc anymore.

comment:6 follow-up: Changed 3 years ago by rmk

It is still the same with current git head. Where do I put -flags showall? I tried both:

ffmpeg -ss 0.96 -flags showall -i gh2-50i-psf.mts -an -sn -vframes 1 out096-showall.jpg

and

ffmpeg -ss 0.96 -i gh2-50i-psf.mts -flags showall -an -sn -vframes 1 out096-showall.jpg

and both fail with errors like:

Unable to parse option value "showall" ...

comment:7 in reply to: ↑ 6 ; follow-up: Changed 3 years ago by cehoyos

Replying to rmk:

It is still the same with current git head. Where do I put -flags showall? I tried both:

ffmpeg -ss 0.96 -flags showall -i gh2-50i-psf.mts -an -sn -vframes 1 out096-showall.jpg

It is actually -flags2 showall

comment:8 in reply to: ↑ 7 Changed 3 years ago by rmk

Replying to cehoyos:

Replying to rmk:

It is still the same with current git head. Where do I put -flags showall? I tried both:

ffmpeg -ss 0.96 -flags showall -i gh2-50i-psf.mts -an -sn -vframes 1 out096-showall.jpg

It is actually -flags2 showall

OK, output is different. The frame is broken in a way that looks as if it was composed using the wrong reference frame.

comment:9 Changed 2 years ago by cehoyos

  • Keywords h264 seek added
Note: See TracTickets for help on using tickets.