Opened 23 months ago

Last modified 11 months ago

#5000 reopened defect

Skipping frames for GoPro videos recorder with Ambarella

Reported by: Den-ffmpeg Owned by:
Priority: important Component: undetermined
Version: git-master Keywords: h264 regression
Cc: michael Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Hi! I am trying to work with GoPro? videos made by Ambarella AVC. I tried to convert any video from that camera and output video was lagging.

The problem is old, it was reproduced in version 2.8.1 and has been existing at least from version 2.2.4

How to reproduce:

ffmpeg -i "D:\videos\h264_gopro_ambarella.mp4" -vcodec h264 "D:\videos\lags_on_output_video.avi"
ffmpeg version 2.8.1 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 5.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b
 --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpe
g --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis
--enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enable-zlib
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100

Attachments (2)

ffmpeg-20151110-094855.log (106.6 KB) - added by Den-ffmpeg 22 months ago.
console output
ffmpeg-20151127-165612(mpeg4).log (28.1 KB) - added by Den-ffmpeg 22 months ago.

Download all attachments as: .zip

Change History (18)

comment:1 Changed 23 months ago by Den-ffmpeg

Input and ouput videos:
http://www.datafilehost.com/d/314e84e3
http://www.datafilehost.com/d/dc890fb6

Firstly, I thought the problem was related to the conversion. However, the ffplay plays the original video with lags as well. So I think, the problem is in decoder. VLC and Media Player Classic play video with lags too. But QuickTime? and Windows Media Player play the video correctly.

Last edited 23 months ago by Den-ffmpeg (previous) (diff)

comment:2 Changed 23 months ago by cehoyos

Please test current FFmpeg git head and please provide the command line that allows to reproduce the issue together with the complete, uncut console output to make this a valid ticket.

comment:3 Changed 23 months ago by Den-ffmpeg

Thanks for responding! Here is a full log from commit a5202bc made by Ganesh Ajjanagadde:
http://www.datafilehost.com/d/fe621ab2

The command line is specified in log, but I'll write it here too:

ffmpeg -i "D:\videos\h264_gopro_ambarella.mp4" -vcodec h264 "D:\videos\lags_on_output_video.avi"
Last edited 23 months ago by Den-ffmpeg (previous) (diff)

comment:4 Changed 23 months ago by cehoyos

  • Resolution set to worksforme
  • Status changed from new to closed

The output file you uploaded works fine here, please reopen this ticket if you can provide command line together with the complete, uncut console output and if you can explain what the issue is.

comment:5 Changed 22 months ago by Den-ffmpeg

  • Resolution worksforme deleted
  • Status changed from closed to reopened

Did you compare the input and output file? I definitely see the difference between them. Please, try to watch them again, for example, in QuickTime? or Windows Media Player. Did you look through the attached log? There are command line and uncut console output. And you might see something like that "[h264 @ 05ccd4e0] no picture ooo" there.

I think, the problem is in frames' order. Specified line comes from \libavcodec\h264.c

    if (!out_of_order && pics > h->avctx->has_b_frames) {
        h->next_output_pic = out;
        if (out_idx == 0 && h->delayed_pic[0] && (h->delayed_pic[0]->f->key_frame || h->delayed_pic[0]->mmco_reset)) {
            h->next_outputed_poc = INT_MIN;
        } else
            h->next_outputed_poc = out->poc;
    } else {
        av_log(h->avctx, AV_LOG_DEBUG, "no picture %s\n", out_of_order ? "ooo" : "");
    }

comment:6 Changed 22 months ago by cehoyos

Please provide the command line that allows to reproduce the issue together with its complete, uncut console output here in the bug tracker, do not use external resources.
Please explain what differences you see when "comparing" the output files: Can the difference also be seen with vlc, MPlayer or FFplay?

Changed 22 months ago by Den-ffmpeg

console output

comment:7 Changed 22 months ago by Den-ffmpeg

Sorry for incomprehension, done.
Command line:

ffmpeg -i "D:\\videos\\h264_gopro_ambarella.mp4" -vcodec h264 "D:\\videos\\lags_on_output_video.avi"

Full log in attach

I see that the output video is lagging. If you open both videos, for example, in QuickTime?, you will see the difference. On output video is more shaking, that effect is created by the fact, that about 25% frames were skipped. On original file the video is more smooth.

No, there is no difference by playing it with VLC, MPlayer or FFplay (also with Media Player Classic). It is explained that these players use the same ffmpeg h264 decoder. But on QuickTime? or Windows Media Player the difference is obvious.

comment:8 Changed 22 months ago by Den-ffmpeg

Could you confirm the problem exists? Will there be any action to fix it?

comment:9 Changed 22 months ago by cehoyos

Unfortunately I am unable to reproduce any issue. Please test with -vcodec mpeg4 -qscale 2 to rule out an issue with an external library (x264).

Changed 22 months ago by Den-ffmpeg

comment:10 Changed 22 months ago by Den-ffmpeg

Have the same problem. Full log is attached. Please note that the problem is in decoder, I can see message from it again:

[h264 @ 05680900] no picture ooo

comment:11 Changed 22 months ago by carriganjohn68

Hi! I have the same issue: the output video is strongly lagging.
I have a lot of GoPro? video files from my customers with the same lagging problem as described in that ticket. I’d like to mention that the popularity of GoPro? cameras is growing constantly and it would be great if you fix it asap.

comment:12 Changed 22 months ago by cehoyos

I cannot reproduce an issue with the sample h264_gopro_ambarella.mp4
Please test with current FFmpeg git head (not a month-old version) and provide command line including complete, uncut console output. Please do not use an external library (x264) if this is not necessary to reproduce the issue.

$ md5sum h264_gopro_ambarella.mp4
b7bfb6524ae5cb74ac38f21ae8245c44  h264_gopro_ambarella.mp4
$ ffmpeg -i h264_gopro_ambarella.mp4 -qscale 2 out.avi
ffmpeg version N-77160-g9aebea0 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      55. 10.100 / 55. 10.100
  libavcodec     57. 17.100 / 57. 17.100
  libavformat    57. 19.100 / 57. 19.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6. 20.100 /  6. 20.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'h264_gopro_ambarella.mp4':
  Metadata:
    major_brand     : avc1
    minor_version   : 0
    compatible_brands: avc1isom
    creation_time   : 2013-01-05 15:25:38
  Duration: 00:00:16.48, start: 0.000000, bitrate: 12270 kb/s
    Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 12126 kb/s, 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc (default)
    Metadata:
      creation_time   : 2013-01-05 15:25:38
      handler_name    :  Ambarella AVC
      encoder         : Ambarella AVC encoder
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2013-01-05 15:25:38
      handler_name    :  Ambarella AAC
Please use -q:a or -q:v, -qscale is ambiguous
Output #0, avi, to 'out.avi':
  Metadata:
    major_brand     : avc1
    minor_version   : 0
    compatible_brands: avc1isom
    ISFT            : Lavf57.19.100
    Stream #0:0(eng): Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 59.94 fps, 59.94 tbn, 59.94 tbc (default)
    Metadata:
      creation_time   : 2013-01-05 15:25:38
      handler_name    :  Ambarella AVC
      encoder         : Lavc57.17.100 mpeg4
    Stream #0:1(eng): Audio: ac3 ([0] [0][0] / 0x2000), 48000 Hz, stereo, fltp, 192 kb/s (default)
    Metadata:
      creation_time   : 2013-01-05 15:25:38
      handler_name    :  Ambarella AAC
      encoder         : Lavc57.17.100 ac3
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mpeg4 (native))
  Stream #0:1 -> #0:1 (aac (native) -> ac3 (native))
Press [q] to stop, [?] for help
frame=  741 fps=284 q=2.0 Lsize=   31664kB time=00:00:16.48 bitrate=15736.9kbits/s
video:31233kB audio:386kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.143361%

comment:13 follow-up: Changed 22 months ago by cehoyos

  • Keywords h264 regression added
  • Priority changed from normal to important
  • Reproduced by developer set
  • Version changed from unspecified to git-master

Please ignore, I can reproduce the issue now, will add more information.

comment:14 in reply to: ↑ 13 Changed 21 months ago by michael

  • Cc michael added

Replying to cehoyos:

Please ignore, I can reproduce the issue now, will add more information.

cant reproduce either, how did you reproduce it ?

comment:15 Changed 12 months ago by richardpl

No samples, no way reproduce this.

comment:16 Changed 11 months ago by alex_hitman

Hi everyone!

I'v done some research of this bug.

Sample video has 988 video frames, but h264 decoder outputs only 741 (every 4th is dropped).
Header has VUI parameters with bitstream restrictions, where num_reordered_frames == 1.
But pts is differ from dts by 2. So decoder drops frames because it does not increase reordered buffer to 2 due to bitstream restrictions.

h264.c:decode_postinit()

    } else if(h->avctx->has_b_frames < out_of_order && !sps->bitstream_restriction_flag){
        av_log(h->avctx, AV_LOG_INFO, "Increasing reorder buffer to %d\n", out_of_order);
        h->avctx->has_b_frames = out_of_order;
    }

Here has_b_frames == num_reordered_frames because bitstream_restriction_flag is set.

After dropping VUI bitstream restrictions decoder outputs all frames.

So, the question is what to do.

h264 standard says that VUI parameters are optional and in our case they are probably wrong.
But may exists other cases where it can be helpful.

May be decoder can do more soft checking. For example, use max_dec_frame_buffering. From standard:

The value of num_reorder_frames shall be in the range of 0 to max_dec_frame_buffering,
inclusive. When the num_reorder_frames syntax element is not present,
the value of num_reorder_frames value shall be inferred to be equal
to max_dec_frame_buffering.

So we can trace warning but drop the frames only after reaching max_dec_frame_buffering.

Or may be don't use num_reordered_frames at all. Seems player are not using ffmpeg for decoding (wmp, quicktime), don't pay much attention to bitstream restrictions.

Reuploaded file: https://www.datafilehost.com/d/7d7b159e

Note: See TracTickets for help on using tickets.