Opened 3 weeks ago

Last modified 3 weeks ago

#6869 new defect

h264_mp4toannexb only inserts SPS/PPS in front of IDR frames

Reported by: mkver Owned by:
Priority: normal Component: avcodec
Version: unspecified Keywords: h264
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

The h264_mp4toannexb bitstream filter currently does not insert the SPS/PPS from the extradata in front of every keyframe, but only in front of IDR frames (at least that's what I observed). This can lead to unplayable files. For this consider a H.264 track whose SPS/PPS are only transmitted out of band and that doesn't have IDR frames, but only I frames with recovery point SEI messages as keyframes. If such a file is remuxed to a transport stream, it doesn't contain any SPS and PPS at all and is therefore undecodeable.
I attach a sample "IDR,I,I,I,IDR.mkv" whose keyframes are (in that order) an IDR frame, three non-IDR I frames and an IDR frame (it also contains non-keyframes). If one cuts the beginning and the end away one gets a file "I,I,I.mkv" whose only keyframes are the three non-IDR I frames.

% ffmpeg -report -i I,I,I.mkv -c copy -bsf:v h264_mp4toannexb I,I,I.ts
ffmpeg started on 2017-11-25 at 22:09:44
Report written to "ffmpeg-20171125-220944.log"
ffmpeg version N-89212-ga60b2425c3 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      56.  2.100 / 56.  2.100
  libavcodec     58.  3.105 / 58.  3.105
  libavformat    58.  2.102 / 58.  2.102
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  2.102 /  7.  2.102
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, matroska,webm, from 'I,I,I.mkv':
  Metadata:
    encoder         : libebml v1.3.5 + libmatroska v1.4.8
    creation_time   : 2017-11-25T20:04:56.000000Z
  Duration: 00:00:00.17, start: 0.000000, bitrate: 303 kb/s
    Stream #0:0(eng): Video: h264 (High), yuv420p(progressive), 640x480, SAR 1:1 DAR 4:3, 24 fps, 24 tbr, 1k tbn, 48 tbc (default)
    Metadata:
      BPS             : 17772
      BPS-eng         : 17772
      DURATION        : 00:00:00.167000000
      DURATION-eng    : 00:00:00.167000000
      NUMBER_OF_FRAMES: 4
      NUMBER_OF_FRAMES-eng: 4
      NUMBER_OF_BYTES : 371
      NUMBER_OF_BYTES-eng: 371
      _STATISTICS_WRITING_APP: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2017-11-25 20:04:56
      _STATISTICS_WRITING_DATE_UTC-eng: 2017-11-25 20:04:56
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Output #0, mpegts, to 'I,I,I.ts':
  Metadata:
    encoder         : Lavf58.2.102
    Stream #0:0(eng): Video: h264 (High), yuv420p(progressive), 640x480 [SAR 1:1 DAR 4:3], q=2-31, 24 fps, 24 tbr, 90k tbn, 1k tbc (default)
    Metadata:
      BPS             : 17772
      BPS-eng         : 17772
      DURATION        : 00:00:00.167000000
      DURATION-eng    : 00:00:00.167000000
      NUMBER_OF_FRAMES: 4
      NUMBER_OF_FRAMES-eng: 4
      NUMBER_OF_BYTES : 371
      NUMBER_OF_BYTES-eng: 371
      _STATISTICS_WRITING_APP: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2017-11-25 20:04:56
      _STATISTICS_WRITING_DATE_UTC-eng: 2017-11-25 20:04:56
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
frame=    4 fps=0.0 q=-1.0 Lsize=       1kB time=00:00:00.04 bitrate= 256.7kbits/s speed=N/A    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 257.608704%

Given that "I,I,I.mkv" doesn't have keyframes, "I,I,I.ts" is undecodeable as already explained:

ffmpeg -report -f mpegts -i I,I,I.ts -f null -
ffmpeg started on 2017-11-25 at 22:11:37
Report written to "ffmpeg-20171125-221137.log"
ffmpeg version N-89212-ga60b2425c3 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      56.  2.100 / 56.  2.100
  libavcodec     58.  3.105 / 58.  3.105
  libavformat    58.  2.102 / 58.  2.102
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  2.102 /  7.  2.102
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
[h264 @ 0000000000168260] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000000000168260] decode_slice_header error
[h264 @ 0000000000168260] no frame!
[h264 @ 0000000000168260] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000000000168260] decode_slice_header error
[h264 @ 0000000000168260] no frame!
[h264 @ 0000000000168260] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000000000168260] decode_slice_header error
[h264 @ 0000000000168260] no frame!
[h264 @ 0000000000168260] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000000000168260] decode_slice_header error
[h264 @ 0000000000168260] no frame!
[mpegts @ 00000000001468e0] decoding for stream 0 failed
[mpegts @ 00000000001468e0] Could not find codec parameters for stream 0 (Video: h264 ([27][0][0][0] / 0x001B), none): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, mpegts, from 'I,I,I.ts':
  Duration: 00:00:00.17, start: 1.483333, bitrate: 63 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), none, 24 tbr, 90k tbn, 180k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
[NULL @ 0000000000168260] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000000000177c20] non-existing PPS 0 referenced
[h264 @ 0000000000177c20] decode_slice_header error
[h264 @ 0000000000177c20] no frame!
[NULL @ 0000000000168260] non-existing PPS 0 referenced
[h264 @ 0000000000181480] non-existing PPS 0 referenced
[h264 @ 0000000000181480] decode_slice_header error
[h264 @ 0000000000181480] no frame!
[NULL @ 0000000000168260] non-existing PPS 0 referenced
[h264 @ 00000000001a79a0] non-existing PPS 0 referenced
[h264 @ 00000000001a79a0] decode_slice_header error
[h264 @ 00000000001a79a0] no frame!
[h264 @ 00000000021791c0] non-existing PPS 0 referenced
[h264 @ 00000000021791c0] decode_slice_header error
[h264 @ 00000000021791c0] no frame!
Error while decoding stream #0:0: Invalid data found when processing input
    Last message repeated 3 times
Finishing stream 0:0 without any data written to it.
Nothing was written into output file 0 (pipe:), because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=N/A time=-577014:32:22.77 bitrate=N/A speed=N/A    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
Conversion failed!

I also used the h264_mp4toannexb bitstream filter on "IDR,I,I,I,IDR.mkv" to check its output:

ffmpeg -report -i IDR,I,I,I,IDR.mkv -c copy -bsf:v h264_mp4toannexb IDR,I,I,I,IDR.2.mkv
ffmpeg started on 2017-11-25 at 21:09:42
Report written to "ffmpeg-20171125-210942.log"
ffmpeg version N-89212-ga60b2425c3 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      56.  2.100 / 56.  2.100
  libavcodec     58.  3.105 / 58.  3.105
  libavformat    58.  2.102 / 58.  2.102
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  2.102 /  7.  2.102
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, matroska,webm, from 'IDR,I,I,I,IDR.mkv':
  Metadata:
    encoder         : libebml v1.3.5 + libmatroska v1.4.8
    creation_time   : 2017-11-25T20:07:54.000000Z
  Duration: 00:00:00.58, start: 0.000000, bitrate: 102 kb/s
    Stream #0:0(eng): Video: h264 (High), yuv420p(progressive), 640x480, SAR 1:1 DAR 4:3, 24 fps, 24 tbr, 1k tbn, 48 tbc (default)
    Metadata:
      BPS             : 19835
      BPS-eng         : 19835
      DURATION        : 00:00:00.584000000
      DURATION-eng    : 00:00:00.584000000
      NUMBER_OF_FRAMES: 14
      NUMBER_OF_FRAMES-eng: 14
      NUMBER_OF_BYTES : 1448
      NUMBER_OF_BYTES-eng: 1448
      _STATISTICS_WRITING_APP: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2017-11-25 20:07:54
      _STATISTICS_WRITING_DATE_UTC-eng: 2017-11-25 20:07:54
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Output #0, matroska, to 'IDR,I,I,I,IDR.2.mkv':
  Metadata:
    encoder         : Lavf58.2.102
    Stream #0:0(eng): Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 640x480 [SAR 1:1 DAR 4:3], q=2-31, 24 fps, 24 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      BPS             : 19835
      BPS-eng         : 19835
      DURATION        : 00:00:00.584000000
      DURATION-eng    : 00:00:00.584000000
      NUMBER_OF_FRAMES: 14
      NUMBER_OF_FRAMES-eng: 14
      NUMBER_OF_BYTES : 1448
      NUMBER_OF_BYTES-eng: 1448
      _STATISTICS_WRITING_APP: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v18.0.0 ('Apricity') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2017-11-25 20:07:54
      _STATISTICS_WRITING_DATE_UTC-eng: 2017-11-25 20:07:54
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
frame=   14 fps=0.0 q=-1.0 Lsize=       3kB time=00:00:00.45 bitrate=  54.4kbits/s speed=N/A    
video:1kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 105.460526%

The check mentioned above was done with the "trace_headers" bitstream filters. I'll attach the logs.
Notice that my findings are based upon the latest Zeranoe build. I hope this to be acceptable.

Attachments (7)

IDR,I,I,I,IDR.mkv (7.3 KB) - added by mkver 3 weeks ago.
The file described above
I,I,I.mkv (6.2 KB) - added by mkver 3 weeks ago.
A Matroska file without IDR frames.
IDR,I,I,I,IDR.headers.txt (134.0 KB) - added by mkver 3 weeks ago.
The output of the trace_headers bitstream filter for the H.264 track inside "IDR,I,I,I,IDR.mkv".
IDR,I,I,I,IDR.2.headers.txt (151.3 KB) - added by mkver 3 weeks ago.
Output of the trace_headers bitstream filter for the file "IDR,I,I,I,IDR.2.mkv" that is created by applying the h264_mp4toannexb bitstream filter to "IDR,I,I,I,IDR.mkv".
ffmpeg-20171125-210942.log (8.1 KB) - added by mkver 3 weeks ago.
The log for the creation of "IDR,I,I,I,IDR.2.mkv". Not really interesting.
ffmpeg-20171125-220944.log (6.3 KB) - added by mkver 3 weeks ago.
The log for creation of "I,I,I.ts".
ffmpeg-20171125-221137.log (18.8 KB) - added by mkver 3 weeks ago.
The log for the failed decode attempt for "I,I,I.ts".

Download all attachments as: .zip

Change History (8)

Changed 3 weeks ago by mkver

The file described above

Changed 3 weeks ago by mkver

A Matroska file without IDR frames.

Changed 3 weeks ago by mkver

The output of the trace_headers bitstream filter for the H.264 track inside "IDR,I,I,I,IDR.mkv".

Changed 3 weeks ago by mkver

Output of the trace_headers bitstream filter for the file "IDR,I,I,I,IDR.2.mkv" that is created by applying the h264_mp4toannexb bitstream filter to "IDR,I,I,I,IDR.mkv".

Changed 3 weeks ago by mkver

The log for the creation of "IDR,I,I,I,IDR.2.mkv". Not really interesting.

Changed 3 weeks ago by mkver

The log for creation of "I,I,I.ts".

Changed 3 weeks ago by mkver

The log for the failed decode attempt for "I,I,I.ts".

comment:1 Changed 3 weeks ago by cehoyos

  • Keywords h264 added; SPS PPS removed
Note: See TracTickets for help on using tickets.