Opened 3 years ago

Last modified 2 years ago

#8395 open defect

SPS/PPS with every slice of an IDR frame

Reported by: Zied Aouina Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: h264
Cc: ziaouina@microsoft.com, stesli@microsoft.com Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: yes

Description

Summary of the bug:
The ffmpeg H264 decoding fails whenever there a input stream where SPS/PPS is sent along every slice of an IDR frame. It seems like the decoder treats every slice as a single frame but incomplete and try to perform some error concealment, and of course the followed P frames get corrupted.
Potential solution is to ignore SPS/PPSs coming in mid IDR frame or Non IDR frame; ie accept SPS/PPS with the first IDR slice only.
How to reproduce:

% ffplay h264_sps_pps_per_slice.h264
ffplay version N-91273-g67747c89ad Copyright (c) 2003-2018 the FFmpeg developers
  built with gcc 7.3.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
  libavutil      56. 18.102 / 56. 18.102
  libavcodec     58. 20.101 / 58. 20.101
  libavformat    58. 17.100 / 58. 17.100
  libavdevice    58.  4.101 / 58.  4.101
  libavfilter     7. 25.100 /  7. 25.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
[h264 @ 000001d54b640080] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b640080] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b640080] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
Input #0, h264, from 'C:/work/video/Zexy_Salavaness_source.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1200k tbn, 60 tbc
[h264 @ 000001d552b78200] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552849800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552ad2800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c64440] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c648c0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c3a040] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b65a000] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c64440] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c648c0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c648c0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c3a040] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c58ac0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552ad2800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b6d9c40] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b65a000] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c3a040] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c58ac0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552b78200] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552849800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552ad2800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b6d9c40] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c58ac0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552b78200] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552849800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552b78200] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552849800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552ad2800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c64440] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c648c0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c3a040] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b65a000] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c64440] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c648c0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552ad2800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b6d9c40] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b65a000] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b6d9c40] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b65a000] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c64440] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c648c0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c3a040] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c58ac0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c3a040] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c58ac0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552b78200] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c58ac0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552b78200] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552849800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame

Attachments (2)

h264_sps_pps_per_slice.h264 (2.4 MB ) - added by Zied Aouina 3 years ago.
Sample h264 bitstream with SPS/PPS attached to every IDR slice
h264_sps_pps_per_idr_slice.png (92.0 KB ) - added by Zied Aouina 3 years ago.
Analysis of h264 bitstream with SPS/PPS attached to every IDR slice

Change History (7)

comment:1 by mkver, 3 years ago

  1. You should upload a sample file.
  2. The parser explicitly treats the occurence of an SEI, an SPS, a PPS or an AUD as the beginning of a new access unit if a video slice has already been encountered. This is actually against the spec.

by Zied Aouina, 3 years ago

Attachment: h264_sps_pps_per_slice.h264 added

Sample h264 bitstream with SPS/PPS attached to every IDR slice

by Zied Aouina, 3 years ago

Analysis of h264 bitstream with SPS/PPS attached to every IDR slice

comment:2 by mkver, 3 years ago

The only problem here seems to be how the parser finds frame ends: If one remuxes this file to Matroska using MKVToolNix (which doesn't have this problem since the fixes to this bug), this code isn't executed at all and decoding is fine.

Last edited 3 years ago by mkver (previous) (diff)

comment:3 by Zied Aouina, 3 years ago

Based the MKVToolNix fix bug #2179 they stopped flushing incomplete frames whenever a new SPS/PPS comes;
https://gitlab.com/mbunkus/mkvtoolnix/commit/df794a1e19cdcb4fdd191327509265c67993df00

Last edited 3 years ago by Carl Eugen Hoyos (previous) (diff)

comment:4 by Carl Eugen Hoyos, 3 years ago

Component: ffplayavcodec
Keywords: h264 added; Decoding fails removed
Version: 4.2git-master

comment:5 by Balling, 2 years ago

Status: newopen

Why not apply bugfix from MKVToolNix?

Note: See TracTickets for help on using tickets.