Opened 4 years ago

Last modified 4 years ago

#8395 open defect

SPS/PPS with every slice of an IDR frame

Reported by: Zied Aouina Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: h264
Cc: ziaouina@microsoft.com, stesli@microsoft.com Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: yes

Description

Summary of the bug:
The ffmpeg H264 decoding fails whenever there a input stream where SPS/PPS is sent along every slice of an IDR frame. It seems like the decoder treats every slice as a single frame but incomplete and try to perform some error concealment, and of course the followed P frames get corrupted.
Potential solution is to ignore SPS/PPSs coming in mid IDR frame or Non IDR frame; ie accept SPS/PPS with the first IDR slice only.
How to reproduce:

% ffplay h264_sps_pps_per_slice.h264
ffplay version N-91273-g67747c89ad Copyright (c) 2003-2018 the FFmpeg developers
  built with gcc 7.3.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
  libavutil      56. 18.102 / 56. 18.102
  libavcodec     58. 20.101 / 58. 20.101
  libavformat    58. 17.100 / 58. 17.100
  libavdevice    58.  4.101 / 58.  4.101
  libavfilter     7. 25.100 /  7. 25.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
[h264 @ 000001d54b640080] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b640080] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b640080] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
Input #0, h264, from 'C:/work/video/Zexy_Salavaness_source.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1200k tbn, 60 tbc
[h264 @ 000001d552b78200] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552849800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552ad2800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c64440] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c648c0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c3a040] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b65a000] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c64440] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c648c0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c648c0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c3a040] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c58ac0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552ad2800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b6d9c40] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b65a000] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c3a040] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c58ac0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552b78200] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552849800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552ad2800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b6d9c40] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c58ac0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552b78200] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552849800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552b78200] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552849800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552ad2800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c64440] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c648c0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c3a040] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b65a000] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c64440] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c648c0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552ad2800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b6d9c40] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b65a000] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b6d9c40] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d54b65a000] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c64440] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c648c0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c3a040] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c58ac0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c3a040] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c58ac0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552b78200] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552c58ac0] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552b78200] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame
[h264 @ 000001d552849800] concealing 2400 DC, 2400 AC, 2400 MV errors in I frame

Attachments (2)

h264_sps_pps_per_slice.h264​ (2.4 MB ) - added by Zied Aouina 4 years ago.
Sample h264 bitstream with SPS/PPS attached to every IDR slice
h264_sps_pps_per_idr_slice.png​ (92.0 KB ) - added by Zied Aouina 4 years ago.
Analysis of h264 bitstream with SPS/PPS attached to every IDR slice

Change History (7)

comment:1 by mkver, 4 years ago

  1. You should upload a sample file.
  2. The parser explicitly treats the occurence of an SEI, an SPS, a PPS or an AUD as the beginning of a new access unit if a video slice has already been encountered. This is actually against the spec.

by Zied Aouina, 4 years ago

Attachment: h264_sps_pps_per_slice.h264​ added

Sample h264 bitstream with SPS/PPS attached to every IDR slice

by Zied Aouina, 4 years ago

Attachment: h264_sps_pps_per_idr_slice.png​ added

Analysis of h264 bitstream with SPS/PPS attached to every IDR slice

comment:2 by mkver, 4 years ago

The only problem here seems to be how the parser finds frame ends: If one remuxes this file to Matroska using MKVToolNix (which doesn't have this problem since ​the fixes to this bug), this code isn't executed at all and decoding is fine.

Last edited 4 years ago by mkver (previous) (diff)

comment:3 by Zied Aouina, 4 years ago

Based the MKVToolNix fix bug #2179 they stopped flushing incomplete frames whenever a new SPS/PPS comes;
​https://gitlab.com/mbunkus/mkvtoolnix/commit/df794a1e19cdcb4fdd191327509265c67993df00

Last edited 4 years ago by Carl Eugen Hoyos (previous) (diff)

comment:4 by Carl Eugen Hoyos, 4 years ago

Component: ffplay → avcodec
Keywords: h264 added; Decoding fails removed
Version: 4.2 → git-master

comment:5 by Balling, 4 years ago

Status: new → open

Why not apply bugfix from MKVToolNix?

Note: See TracTickets for help on using tickets.