Opened 6 years ago

Closed 6 years ago

#6864 closed defect (fixed)

H.264 with multiple PPS - extradata in Matroska not complete upon remuxing

Reported by: mkver Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: h264
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: yes

Description

Hello,

my sample is a transport stream that contains (among other tracks) an H.264 track with one SPS and three PPS with different pic_parameter_set_id values that reference the unique SPS. These SPS and PPS are repeated throughout the stream (in front of every keyframe). When remuxing this track to Matroska, only one of these PPS ends up in the CodecPrivate although all could fit in there (they are compatible as their ids are different). Given that the bitstream still contains all SPS and PPS, the resulting file is playable, but this is suboptimal because there are bitstream filters that rely on correct extradata: Given that the SPS and PPS don't change one could save some bytes by removing these SPS and PPS from the bitstream with the remove_extra bitstream filter, but the resulting file would be unplayable, because only one of the PPS is in the CodecPrivate.
I have used the latest Zeranoe build to test this. I hope this to be acceptable.
How to reproduce:

ffmpeg -report -i multiple.pps.ts -map 0:0 -c copy multiple.pps.mkv
ffmpeg started on 2017-11-24 at 18:45:53
Report written to "ffmpeg-20171124-184553.log"
ffmpeg version N-89212-ga60b2425c3 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      56.  2.100 / 56.  2.100
  libavcodec     58.  3.105 / 58.  3.105
  libavformat    58.  2.102 / 58.  2.102
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  2.102 /  7.  2.102
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
[h264 @ 000000000057a5a0] non-existing PPS 0 referenced
[h264 @ 000000000057a5a0] decode_slice_header error
[h264 @ 000000000057a5a0] no frame!
[mpegts @ 0000000000576940] PES packet size mismatch
    Last message repeated 3 times
Input #0, mpegts, from 'multiple.pps.ts':
  Duration: 00:00:07.08, start: 45067.839433, bitrate: 11848 kb/s
  Program 10352 
    Stream #0:0[0x14d3]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k tbn, 100 tbc
    Stream #0:1[0x14d4](deu): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s
    Stream #0:2[0x14d5](mis): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s
    Stream #0:3[0x14d8](deu): Audio: ac3 ([6][0][0][0] / 0x0006), 48000 Hz, stereo, fltp, 448 kb/s
    Stream #0:4[0x14d7](deu): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006) (hearing impaired)
    Stream #0:5[0x14d6](deu): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
Output #0, matroska, to 'multiple.pps.mkv':
  Metadata:
    encoder         : Lavf58.2.102
    Stream #0:0: Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 50 fps, 50 tbr, 1k tbn, 90k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
frame=  257 fps=0.0 q=-1.0 Lsize=    8219kB time=00:00:05.66 bitrate=11893.4kbits/s speed=  59x    
video:8216kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.037966%

I also attach the output of the trace_headers bitstream-filter for the transport stream and the Matroska file. I want to draw your attention to the fact that the trace_headers output for the transport stream reports all three PPS as extradata; the extradata-section of the trace_headers-output for the Matroska file only shows one PPS as extradata (the one with the id 2 (which is the highest id of the three PPS)). So maybe the problem is in the component that converts from annex B H.264 to the MP4 format H.264. Maybe it is not made to put multiple different, but compatible SPS and PPS in the CodecPrivate.

Attachments (7)

header.ts.txt (428.9 KB ) - added by mkver 6 years ago.
This is the output of the trace_headers bitstream filter for the H.264 track in the transport stream.
header.mkv.txt (397.3 KB ) - added by mkver 6 years ago.
This is the output of the trace_headers bitstream filter for the Matroska file created by remuxing the H.264 track of the transport stream.
Multiple.PPS.ts.001 (2.0 MB ) - added by mkver 6 years ago.
First part of the input file.
Multiple.PPS.ts.002 (2.0 MB ) - added by mkver 6 years ago.
Second part of the input file.
Multiple.PPS.ts.003 (2.0 MB ) - added by mkver 6 years ago.
Third part of the input file.
Multiple.PPS.ts.004 (2.0 MB ) - added by mkver 6 years ago.
Fourth part of the input file.
Multiple.PPS.ts.005 (2.0 MB ) - added by mkver 6 years ago.
Last part of the input file.

Change History (9)

by mkver, 6 years ago

Attachment: header.ts.txt added

This is the output of the trace_headers bitstream filter for the H.264 track in the transport stream.

by mkver, 6 years ago

Attachment: header.mkv.txt added

This is the output of the trace_headers bitstream filter for the Matroska file created by remuxing the H.264 track of the transport stream.

by mkver, 6 years ago

Attachment: Multiple.PPS.ts.001 added

First part of the input file.

by mkver, 6 years ago

Attachment: Multiple.PPS.ts.002 added

Second part of the input file.

by mkver, 6 years ago

Attachment: Multiple.PPS.ts.003 added

Third part of the input file.

by mkver, 6 years ago

Attachment: Multiple.PPS.ts.004 added

Fourth part of the input file.

by mkver, 6 years ago

Attachment: Multiple.PPS.ts.005 added

Last part of the input file.

comment:1 by James, 6 years ago

Analyzed by developer: set
Component: undeterminedavformat
Keywords: h264 added; SPS PPS removed
Status: newopen
Version: unspecifiedgit-master

The avcc writing code in lavf is explicitly writing only the last sps and the last pps it finds in annexb style extradata. It should indeed instead write all.

comment:2 by James, 6 years ago

Resolution: fixed
Status: openclosed
Note: See TracTickets for help on using tickets.