Opened 11 months ago

Last modified 11 months ago

#6869 new defect

h264_mp4toannexb only inserts SPS/PPS in front of IDR frames

Reported by: mkver Owned by:
Priority: normal Component: avcodec
Version: unspecified Keywords: h264
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

The h264_mp4toannexb bitstream filter currently does not insert the SPS/PPS from the extradata in front of every keyframe, but only in front of IDR frames (at least that's what I observed). This can lead to unplayable files. For this consider a H.264 track whose SPS/PPS are only transmitted out of band and that doesn't have IDR frames, but only I frames with recovery point SEI messages as keyframes. If such a file is remuxed to a transport stream, it doesn't contain any SPS and PPS at all and is therefore undecodeable.
I attach a sample "IDR,I,I,I,IDR.mkv" whose keyframes are (in that order) an IDR frame, three non-IDR I frames and an IDR frame (it also contains non-keyframes). If one cuts the beginning and the end away one gets a file "I,I,I.mkv" whose only keyframes are the three non-IDR I frames.

% ffmpeg -report -i I,I,I.mkv -c copy -bsf:v h264_mp4toannexb I,I,I.ts
ffmpeg started on 2017-11-25 at 22:09:44
Report written to "ffmpeg-20171125-220944.log"
ffmpeg version N-89212-ga60b2425c3 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      56.  2.100 / 56.  2.100
  libavcodec     58.  3.105 / 58.  3.105
  libavformat    58.  2.102 / 58.  2.102
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  2.102 /  7.  2.102
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, matroska,webm, from 'I,I,I.mkv':
  Metadata:
    encoder         : libebml v1.3.5 + libmatroska v1.4.8
    creation_time   : 2017-11-25T20:04:56.000000Z
  Duration: 00:00:00.17, start: 0.000000, bitrate: 303 kb/s
    Stream #0:0(eng): Video: h264 (High), yuv420p(progressive), 640x480, SAR 1:1 DAR 4:3, 24 fps, 24 tbr, 1k tbn, 48 tbc (default)
    Metadata:
      BPS             : 17772
      BPS-eng         : 17772
      DURATION        : 00:00:00.167000000
      DURATION-eng    : 00:00:00.167000000
      NUMBER_OF_FRAMES: 4
      NUMBER_OF_FRAMES-eng: 4
      NUMBER_OF_BYTES : 371
      NUMBER_OF_BYTES-eng: 371
      _STATISTICS_WRITING_APP: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2017-11-25 20:04:56
      _STATISTICS_WRITING_DATE_UTC-eng: 2017-11-25 20:04:56
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Output #0, mpegts, to 'I,I,I.ts':
  Metadata:
    encoder         : Lavf58.2.102
    Stream #0:0(eng): Video: h264 (High), yuv420p(progressive), 640x480 [SAR 1:1 DAR 4:3], q=2-31, 24 fps, 24 tbr, 90k tbn, 1k tbc (default)
    Metadata:
      BPS             : 17772
      BPS-eng         : 17772
      DURATION        : 00:00:00.167000000
      DURATION-eng    : 00:00:00.167000000
      NUMBER_OF_FRAMES: 4
      NUMBER_OF_FRAMES-eng: 4
      NUMBER_OF_BYTES : 371
      NUMBER_OF_BYTES-eng: 371
      _STATISTICS_WRITING_APP: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2017-11-25 20:04:56
      _STATISTICS_WRITING_DATE_UTC-eng: 2017-11-25 20:04:56
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
frame=    4 fps=0.0 q=-1.0 Lsize=       1kB time=00:00:00.04 bitrate= 256.7kbits/s speed=N/A    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 257.608704%

Given that "I,I,I.mkv" doesn't have keyframes, "I,I,I.ts" is undecodeable as already explained:

ffmpeg -report -f mpegts -i I,I,I.ts -f null -
ffmpeg started on 2017-11-25 at 22:11:37
Report written to "ffmpeg-20171125-221137.log"
ffmpeg version N-89212-ga60b2425c3 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      56.  2.100 / 56.  2.100
  libavcodec     58.  3.105 / 58.  3.105
  libavformat    58.  2.102 / 58.  2.102
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  2.102 /  7.  2.102
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
[h264 @ 0000000000168260] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000000000168260] decode_slice_header error
[h264 @ 0000000000168260] no frame!
[h264 @ 0000000000168260] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000000000168260] decode_slice_header error
[h264 @ 0000000000168260] no frame!
[h264 @ 0000000000168260] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000000000168260] decode_slice_header error
[h264 @ 0000000000168260] no frame!
[h264 @ 0000000000168260] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000000000168260] decode_slice_header error
[h264 @ 0000000000168260] no frame!
[mpegts @ 00000000001468e0] decoding for stream 0 failed
[mpegts @ 00000000001468e0] Could not find codec parameters for stream 0 (Video: h264 ([27][0][0][0] / 0x001B), none): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, mpegts, from 'I,I,I.ts':
  Duration: 00:00:00.17, start: 1.483333, bitrate: 63 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), none, 24 tbr, 90k tbn, 180k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
[NULL @ 0000000000168260] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000000000177c20] non-existing PPS 0 referenced
[h264 @ 0000000000177c20] decode_slice_header error
[h264 @ 0000000000177c20] no frame!
[NULL @ 0000000000168260] non-existing PPS 0 referenced
[h264 @ 0000000000181480] non-existing PPS 0 referenced
[h264 @ 0000000000181480] decode_slice_header error
[h264 @ 0000000000181480] no frame!
[NULL @ 0000000000168260] non-existing PPS 0 referenced
[h264 @ 00000000001a79a0] non-existing PPS 0 referenced
[h264 @ 00000000001a79a0] decode_slice_header error
[h264 @ 00000000001a79a0] no frame!
[h264 @ 00000000021791c0] non-existing PPS 0 referenced
[h264 @ 00000000021791c0] decode_slice_header error
[h264 @ 00000000021791c0] no frame!
Error while decoding stream #0:0: Invalid data found when processing input
    Last message repeated 3 times
Finishing stream 0:0 without any data written to it.
Nothing was written into output file 0 (pipe:), because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=N/A time=-577014:32:22.77 bitrate=N/A speed=N/A    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
Conversion failed!

I also used the h264_mp4toannexb bitstream filter on "IDR,I,I,I,IDR.mkv" to check its output:

ffmpeg -report -i IDR,I,I,I,IDR.mkv -c copy -bsf:v h264_mp4toannexb IDR,I,I,I,IDR.2.mkv
ffmpeg started on 2017-11-25 at 21:09:42
Report written to "ffmpeg-20171125-210942.log"
ffmpeg version N-89212-ga60b2425c3 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      56.  2.100 / 56.  2.100
  libavcodec     58.  3.105 / 58.  3.105
  libavformat    58.  2.102 / 58.  2.102
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  2.102 /  7.  2.102
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, matroska,webm, from 'IDR,I,I,I,IDR.mkv':
  Metadata:
    encoder         : libebml v1.3.5 + libmatroska v1.4.8
    creation_time   : 2017-11-25T20:07:54.000000Z
  Duration: 00:00:00.58, start: 0.000000, bitrate: 102 kb/s
    Stream #0:0(eng): Video: h264 (High), yuv420p(progressive), 640x480, SAR 1:1 DAR 4:3, 24 fps, 24 tbr, 1k tbn, 48 tbc (default)
    Metadata:
      BPS             : 19835
      BPS-eng         : 19835
      DURATION        : 00:00:00.584000000
      DURATION-eng    : 00:00:00.584000000
      NUMBER_OF_FRAMES: 14
      NUMBER_OF_FRAMES-eng: 14
      NUMBER_OF_BYTES : 1448
      NUMBER_OF_BYTES-eng: 1448
      _STATISTICS_WRITING_APP: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2017-11-25 20:07:54
      _STATISTICS_WRITING_DATE_UTC-eng: 2017-11-25 20:07:54
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Output #0, matroska, to 'IDR,I,I,I,IDR.2.mkv':
  Metadata:
    encoder         : Lavf58.2.102
    Stream #0:0(eng): Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 640x480 [SAR 1:1 DAR 4:3], q=2-31, 24 fps, 24 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      BPS             : 19835
      BPS-eng         : 19835
      DURATION        : 00:00:00.584000000
      DURATION-eng    : 00:00:00.584000000
      NUMBER_OF_FRAMES: 14
      NUMBER_OF_FRAMES-eng: 14
      NUMBER_OF_BYTES : 1448
      NUMBER_OF_BYTES-eng: 1448
      _STATISTICS_WRITING_APP: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2017-11-25 20:07:54
      _STATISTICS_WRITING_DATE_UTC-eng: 2017-11-25 20:07:54
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
frame=   14 fps=0.0 q=-1.0 Lsize=       3kB time=00:00:00.45 bitrate=  54.4kbits/s speed=N/A    
video:1kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 105.460526%

The check mentioned above was done with the "trace_headers" bitstream filters. I'll attach the logs.
Notice that my findings are based upon the latest Zeranoe build. I hope this to be acceptable.

Attachments (7)

IDR,I,I,I,IDR.mkv (7.3 KB) - added by mkver 11 months ago.
The file described above
I,I,I.mkv (6.2 KB) - added by mkver 11 months ago.
A Matroska file without IDR frames.
IDR,I,I,I,IDR.headers.txt (134.0 KB) - added by mkver 11 months ago.
The output of the trace_headers bitstream filter for the H.264 track inside "IDR,I,I,I,IDR.mkv".
IDR,I,I,I,IDR.2.headers.txt (151.3 KB) - added by mkver 11 months ago.
Output of the trace_headers bitstream filter for the file "IDR,I,I,I,IDR.2.mkv" that is created by applying the h264_mp4toannexb bitstream filter to "IDR,I,I,I,IDR.mkv".
ffmpeg-20171125-210942.log (8.1 KB) - added by mkver 11 months ago.
The log for the creation of "IDR,I,I,I,IDR.2.mkv". Not really interesting.
ffmpeg-20171125-220944.log (6.3 KB) - added by mkver 11 months ago.
The log for creation of "I,I,I.ts".
ffmpeg-20171125-221137.log (18.8 KB) - added by mkver 11 months ago.
The log for the failed decode attempt for "I,I,I.ts".

Download all attachments as: .zip

Change History (8)

Changed 11 months ago by mkver

The file described above

Changed 11 months ago by mkver

A Matroska file without IDR frames.

Changed 11 months ago by mkver

The output of the trace_headers bitstream filter for the H.264 track inside "IDR,I,I,I,IDR.mkv".

Changed 11 months ago by mkver

Output of the trace_headers bitstream filter for the file "IDR,I,I,I,IDR.2.mkv" that is created by applying the h264_mp4toannexb bitstream filter to "IDR,I,I,I,IDR.mkv".

Changed 11 months ago by mkver

The log for the creation of "IDR,I,I,I,IDR.2.mkv". Not really interesting.

Changed 11 months ago by mkver

The log for creation of "I,I,I.ts".

Changed 11 months ago by mkver

The log for the failed decode attempt for "I,I,I.ts".

comment:1 Changed 11 months ago by cehoyos

  • Keywords h264 added; SPS PPS removed
Note: See TracTickets for help on using tickets.