Opened 9 years ago
Last modified 2 years ago
#6869 new defect
h264_mp4toannexb only inserts SPS/PPS in front of IDR frames
| Reported by: | mkver | Owned by: | |
|---|---|---|---|
| Priority: | normal | Component: | avcodec |
| Version: | unspecified | Keywords: | h264 |
| Cc: | Blocked By: | ||
| Blocking: | Reproduced by developer: | no | |
| Analyzed by developer: | no |
Description
The h264_mp4toannexb bitstream filter currently does not insert the SPS/PPS from the extradata in front of every keyframe, but only in front of IDR frames (at least that's what I observed). This can lead to unplayable files. For this consider a H.264 track whose SPS/PPS are only transmitted out of band and that doesn't have IDR frames, but only I frames with recovery point SEI messages as keyframes. If such a file is remuxed to a transport stream, it doesn't contain any SPS and PPS at all and is therefore undecodeable.
I attach a sample "IDR,I,I,I,IDR.mkv" whose keyframes are (in that order) an IDR frame, three non-IDR I frames and an IDR frame (it also contains non-keyframes). If one cuts the beginning and the end away one gets a file "I,I,I.mkv" whose only keyframes are the three non-IDR I frames.
% ffmpeg -report -i I,I,I.mkv -c copy -bsf:v h264_mp4toannexb I,I,I.ts
ffmpeg started on 2017-11-25 at 22:09:44
Report written to "ffmpeg-20171125-220944.log"
ffmpeg version N-89212-ga60b2425c3 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 7.2.0 (GCC)
configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
libavutil 56. 2.100 / 56. 2.100
libavcodec 58. 3.105 / 58. 3.105
libavformat 58. 2.102 / 58. 2.102
libavdevice 58. 0.100 / 58. 0.100
libavfilter 7. 2.102 / 7. 2.102
libswscale 5. 0.101 / 5. 0.101
libswresample 3. 0.101 / 3. 0.101
libpostproc 55. 0.100 / 55. 0.100
Input #0, matroska,webm, from 'I,I,I.mkv':
Metadata:
encoder : libebml v1.3.5 + libmatroska v1.4.8
creation_time : 2017-11-25T20:04:56.000000Z
Duration: 00:00:00.17, start: 0.000000, bitrate: 303 kb/s
Stream #0:0(eng): Video: h264 (High), yuv420p(progressive), 640x480, SAR 1:1 DAR 4:3, 24 fps, 24 tbr, 1k tbn, 48 tbc (default)
Metadata:
BPS : 17772
BPS-eng : 17772
DURATION : 00:00:00.167000000
DURATION-eng : 00:00:00.167000000
NUMBER_OF_FRAMES: 4
NUMBER_OF_FRAMES-eng: 4
NUMBER_OF_BYTES : 371
NUMBER_OF_BYTES-eng: 371
_STATISTICS_WRITING_APP: mkvmerge v18.0.0 ('Apricity') 64-bit
_STATISTICS_WRITING_APP-eng: mkvmerge v18.0.0 ('Apricity') 64-bit
_STATISTICS_WRITING_DATE_UTC: 2017-11-25 20:04:56
_STATISTICS_WRITING_DATE_UTC-eng: 2017-11-25 20:04:56
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Output #0, mpegts, to 'I,I,I.ts':
Metadata:
encoder : Lavf58.2.102
Stream #0:0(eng): Video: h264 (High), yuv420p(progressive), 640x480 [SAR 1:1 DAR 4:3], q=2-31, 24 fps, 24 tbr, 90k tbn, 1k tbc (default)
Metadata:
BPS : 17772
BPS-eng : 17772
DURATION : 00:00:00.167000000
DURATION-eng : 00:00:00.167000000
NUMBER_OF_FRAMES: 4
NUMBER_OF_FRAMES-eng: 4
NUMBER_OF_BYTES : 371
NUMBER_OF_BYTES-eng: 371
_STATISTICS_WRITING_APP: mkvmerge v18.0.0 ('Apricity') 64-bit
_STATISTICS_WRITING_APP-eng: mkvmerge v18.0.0 ('Apricity') 64-bit
_STATISTICS_WRITING_DATE_UTC: 2017-11-25 20:04:56
_STATISTICS_WRITING_DATE_UTC-eng: 2017-11-25 20:04:56
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
frame= 4 fps=0.0 q=-1.0 Lsize= 1kB time=00:00:00.04 bitrate= 256.7kbits/s speed=N/A
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 257.608704%
Given that "I,I,I.mkv" doesn't have keyframes, "I,I,I.ts" is undecodeable as already explained:
ffmpeg -report -f mpegts -i I,I,I.ts -f null -
ffmpeg started on 2017-11-25 at 22:11:37
Report written to "ffmpeg-20171125-221137.log"
ffmpeg version N-89212-ga60b2425c3 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 7.2.0 (GCC)
configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
libavutil 56. 2.100 / 56. 2.100
libavcodec 58. 3.105 / 58. 3.105
libavformat 58. 2.102 / 58. 2.102
libavdevice 58. 0.100 / 58. 0.100
libavfilter 7. 2.102 / 7. 2.102
libswscale 5. 0.101 / 5. 0.101
libswresample 3. 0.101 / 3. 0.101
libpostproc 55. 0.100 / 55. 0.100
[h264 @ 0000000000168260] non-existing PPS 0 referenced
Last message repeated 1 times
[h264 @ 0000000000168260] decode_slice_header error
[h264 @ 0000000000168260] no frame!
[h264 @ 0000000000168260] non-existing PPS 0 referenced
Last message repeated 1 times
[h264 @ 0000000000168260] decode_slice_header error
[h264 @ 0000000000168260] no frame!
[h264 @ 0000000000168260] non-existing PPS 0 referenced
Last message repeated 1 times
[h264 @ 0000000000168260] decode_slice_header error
[h264 @ 0000000000168260] no frame!
[h264 @ 0000000000168260] non-existing PPS 0 referenced
Last message repeated 1 times
[h264 @ 0000000000168260] decode_slice_header error
[h264 @ 0000000000168260] no frame!
[mpegts @ 00000000001468e0] decoding for stream 0 failed
[mpegts @ 00000000001468e0] Could not find codec parameters for stream 0 (Video: h264 ([27][0][0][0] / 0x001B), none): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, mpegts, from 'I,I,I.ts':
Duration: 00:00:00.17, start: 1.483333, bitrate: 63 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), none, 24 tbr, 90k tbn, 180k tbc
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
[NULL @ 0000000000168260] non-existing PPS 0 referenced
Last message repeated 1 times
[h264 @ 0000000000177c20] non-existing PPS 0 referenced
[h264 @ 0000000000177c20] decode_slice_header error
[h264 @ 0000000000177c20] no frame!
[NULL @ 0000000000168260] non-existing PPS 0 referenced
[h264 @ 0000000000181480] non-existing PPS 0 referenced
[h264 @ 0000000000181480] decode_slice_header error
[h264 @ 0000000000181480] no frame!
[NULL @ 0000000000168260] non-existing PPS 0 referenced
[h264 @ 00000000001a79a0] non-existing PPS 0 referenced
[h264 @ 00000000001a79a0] decode_slice_header error
[h264 @ 00000000001a79a0] no frame!
[h264 @ 00000000021791c0] non-existing PPS 0 referenced
[h264 @ 00000000021791c0] decode_slice_header error
[h264 @ 00000000021791c0] no frame!
Error while decoding stream #0:0: Invalid data found when processing input
Last message repeated 3 times
Finishing stream 0:0 without any data written to it.
Nothing was written into output file 0 (pipe:), because at least one of its streams received no packets.
frame= 0 fps=0.0 q=0.0 Lsize=N/A time=-577014:32:22.77 bitrate=N/A speed=N/A
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
Conversion failed!
I also used the h264_mp4toannexb bitstream filter on "IDR,I,I,I,IDR.mkv" to check its output:
ffmpeg -report -i IDR,I,I,I,IDR.mkv -c copy -bsf:v h264_mp4toannexb IDR,I,I,I,IDR.2.mkv
ffmpeg started on 2017-11-25 at 21:09:42
Report written to "ffmpeg-20171125-210942.log"
ffmpeg version N-89212-ga60b2425c3 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 7.2.0 (GCC)
configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
libavutil 56. 2.100 / 56. 2.100
libavcodec 58. 3.105 / 58. 3.105
libavformat 58. 2.102 / 58. 2.102
libavdevice 58. 0.100 / 58. 0.100
libavfilter 7. 2.102 / 7. 2.102
libswscale 5. 0.101 / 5. 0.101
libswresample 3. 0.101 / 3. 0.101
libpostproc 55. 0.100 / 55. 0.100
Input #0, matroska,webm, from 'IDR,I,I,I,IDR.mkv':
Metadata:
encoder : libebml v1.3.5 + libmatroska v1.4.8
creation_time : 2017-11-25T20:07:54.000000Z
Duration: 00:00:00.58, start: 0.000000, bitrate: 102 kb/s
Stream #0:0(eng): Video: h264 (High), yuv420p(progressive), 640x480, SAR 1:1 DAR 4:3, 24 fps, 24 tbr, 1k tbn, 48 tbc (default)
Metadata:
BPS : 19835
BPS-eng : 19835
DURATION : 00:00:00.584000000
DURATION-eng : 00:00:00.584000000
NUMBER_OF_FRAMES: 14
NUMBER_OF_FRAMES-eng: 14
NUMBER_OF_BYTES : 1448
NUMBER_OF_BYTES-eng: 1448
_STATISTICS_WRITING_APP: mkvmerge v18.0.0 ('Apricity') 64-bit
_STATISTICS_WRITING_APP-eng: mkvmerge v18.0.0 ('Apricity') 64-bit
_STATISTICS_WRITING_DATE_UTC: 2017-11-25 20:07:54
_STATISTICS_WRITING_DATE_UTC-eng: 2017-11-25 20:07:54
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Output #0, matroska, to 'IDR,I,I,I,IDR.2.mkv':
Metadata:
encoder : Lavf58.2.102
Stream #0:0(eng): Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 640x480 [SAR 1:1 DAR 4:3], q=2-31, 24 fps, 24 tbr, 1k tbn, 1k tbc (default)
Metadata:
BPS : 19835
BPS-eng : 19835
DURATION : 00:00:00.584000000
DURATION-eng : 00:00:00.584000000
NUMBER_OF_FRAMES: 14
NUMBER_OF_FRAMES-eng: 14
NUMBER_OF_BYTES : 1448
NUMBER_OF_BYTES-eng: 1448
_STATISTICS_WRITING_APP: mkvmerge v18.0.0 ('Apricity') 64-bit
_STATISTICS_WRITING_APP-eng: mkvmerge v18.0.0 ('Apricity') 64-bit
_STATISTICS_WRITING_DATE_UTC: 2017-11-25 20:07:54
_STATISTICS_WRITING_DATE_UTC-eng: 2017-11-25 20:07:54
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
frame= 14 fps=0.0 q=-1.0 Lsize= 3kB time=00:00:00.45 bitrate= 54.4kbits/s speed=N/A
video:1kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 105.460526%
The check mentioned above was done with the "trace_headers" bitstream filters. I'll attach the logs.
Notice that my findings are based upon the latest Zeranoe build. I hope this to be acceptable.
Attachments (7)
Change History (9)
by , 9 years ago
| Attachment: | IDR,I,I,I,IDR.mkv added |
|---|
by , 9 years ago
| Attachment: | IDR,I,I,I,IDR.headers.txt added |
|---|
The output of the trace_headers bitstream filter for the H.264 track inside "IDR,I,I,I,IDR.mkv".
by , 9 years ago
| Attachment: | IDR,I,I,I,IDR.2.headers.txt added |
|---|
Output of the trace_headers bitstream filter for the file "IDR,I,I,I,IDR.2.mkv" that is created by applying the h264_mp4toannexb bitstream filter to "IDR,I,I,I,IDR.mkv".
by , 9 years ago
| Attachment: | ffmpeg-20171125-210942.log added |
|---|
The log for the creation of "IDR,I,I,I,IDR.2.mkv". Not really interesting.
by , 9 years ago
| Attachment: | ffmpeg-20171125-221137.log added |
|---|
The log for the failed decode attempt for "I,I,I.ts".
comment:1 by , 9 years ago
| Keywords: | h264 added; SPS PPS removed |
|---|
comment:2 by , 2 years ago
consider a H.264 track whose SPS/PPS are only transmitted out of band and that doesn't have IDR frames, but only I frames with recovery point SEI messages as keyframes
Well, this is not a valid stream per the spec. Each stream should start with IDR. Anyway, I see nowadays it is almost always inserted.
Note:
See TracTickets
for help on using tickets.
The file described above