Opened 10 years ago

Last modified 10 years ago

#3442 open defect

h264 decoder decodes complete GOP wrong / skips one GOP if stream does not start with SPS/PPS

Reported by: Marco Munderloh Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: h264 mpegts
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug:

I recorded a DVB TS stream. It contains an H264 video stream which first NALU is a B slice, followed by the SPS, PPS NALUs and an IDR frame.

If this stream is played with ffplay (either the TS or the demuxed elementary stream), the decoder seems to get so much confused by the leading B slice that the following valid GOP is completely decoded erroneous until the next GOP arrives. ffplay skips the broken frames and starts playing at the second GOP by default. One has to specify -flags2 +showall to see the GOP of broken frames.

If the B slice is removed from the stream, playing starts fine from the first frame/GOP on.

I see that it is intended that ffmpeg tries to decode and conceal every frame coming in, but that should effect only one frame (the B slice) and not the following GOP.

However, I'm not so sure why the B slice is concealed as there is no valid SPS/PPS yet which is mandatory to decode the slice. But that might be a different problem.

Take a h264 stream starting with a B slice, followed by the SPS/PPS/IDR. I uploaded a sample TS file to ftp://upload.ffmpeg.org/MPlayer/incoming/ called DVB_testvideo.ts. One can either play that file or extract the H264 elementary stream and use it directly.

The stream is using PAFF/MBAFF, but I don't know if this is related.

How to reproduce:

% ffplay DVB_testvideo.ts
skips the first valid GOP completely and starts playing from frame 16 onwards.

% ffplay DVB_testvideo.ts -flags2 +showall
starts playing 16 broken frames, then recovers and plays fine.

Now remove the first B slice (skip 17756 from the H264 bitstream or 20304 bytes from the whole TS file).

% ffplay DVB_testvideo.ts
plays fine from the first frame on (15 frames valid frames more then before)

% ffplay DVB_testvideo.ts -flags2 +showall
shows no broken frames anymore


ffmpeg version N-42015-g64e4bd7
built on openSUSE 13.1 (gcc 4.8.1)

Attachments (8)

DVB_testvideo.log (2.5 KB ) - added by Marco Munderloh 10 years ago.
ffplay output for the original TS file
DVB_testvideo_sync.log (1.7 KB ) - added by Marco Munderloh 10 years ago.
ffplay output for the TS file with B slice removed
H264_ES.log (2.0 KB ) - added by Marco Munderloh 10 years ago.
ffplay output for the original H264 elementary stream
H264_ES_sync.log (1.1 KB ) - added by Marco Munderloh 10 years ago.
ffplay output for the H264 elementary stream with B slice removed
stream.lavf.log (42.0 KB ) - added by Marco Munderloh 10 years ago.
h264bitstream analyzer output for original H264 stream
stream.lavf_sync.log (42.0 KB ) - added by Marco Munderloh 10 years ago.
h264bitstream analyzer output for H264 stream with B slice removed
stream.lavf.264 (2.0 MB ) - added by Marco Munderloh 10 years ago.
original H264 bistream, first 2 MiB
stream.lavf_sync.264 (2.0 MB ) - added by Marco Munderloh 10 years ago.
H264 bistream, B slice removed, first 2 MiB

Change History (12)

comment:1 by Marco Munderloh, 10 years ago

I just discovered that

% ffplay DVB_testvideo.ts -flags2 +showall

still shows one broken frame at the beginning of the stream.

by Marco Munderloh, 10 years ago

Attachment: DVB_testvideo.log added

ffplay output for the original TS file

by Marco Munderloh, 10 years ago

Attachment: DVB_testvideo_sync.log added

ffplay output for the TS file with B slice removed

by Marco Munderloh, 10 years ago

Attachment: H264_ES.log added

ffplay output for the original H264 elementary stream

by Marco Munderloh, 10 years ago

Attachment: H264_ES_sync.log added

ffplay output for the H264 elementary stream with B slice removed

comment:2 by Carl Eugen Hoyos, 10 years ago

Keywords: h264 mpegts added; H.264 broken frames PAFF MBAFF removed

comment:3 by Marco Munderloh, 10 years ago

I discovered that I was wrong assuming there is an IDR frame following the SPS/PPS.
The whole stream does actually not contain any IDR but normal I slices only!

This might produce the single broken frame in the beginning that is still
existing even with the B slice removed and might also be responsible for
baffling the decoder.

by Marco Munderloh, 10 years ago

Attachment: stream.lavf.log added

h264bitstream analyzer output for original H264 stream

by Marco Munderloh, 10 years ago

Attachment: stream.lavf_sync.log added

h264bitstream analyzer output for H264 stream with B slice removed

by Marco Munderloh, 10 years ago

Attachment: stream.lavf.264 added

original H264 bistream, first 2 MiB

by Marco Munderloh, 10 years ago

Attachment: stream.lavf_sync.264 added

H264 bistream, B slice removed, first 2 MiB

comment:4 by Carl Eugen Hoyos, 10 years ago

Reproduced by developer: set
Status: newopen

http://thread.gmane.org/gmane.comp.video.mplayer.user/70768

I don't think this is a regression, there are different ways to reproduce the problem, I tested the following two command lines:

$ ffmpeg -skip_initial_bytes 0 -r 10 -i stream.lavf.264 -qscale 2 out1.avi
ffmpeg version N-61165-g742d860 Copyright (c) 2000-2014 the FFmpeg developers
  built on Mar  8 2014 09:01:38 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      52. 66.101 / 52. 66.101
  libavcodec     55. 52.102 / 55. 52.102
  libavformat    55. 33.101 / 55. 33.101
  libavdevice    55. 11.100 / 55. 11.100
  libavfilter     4.  3.100 /  4.  3.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 18.100 /  0. 18.100
  libpostproc    52.  3.100 / 52.  3.100
[h264 @ 0x1e0d960] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0x1e0d960] decode_slice_header error
[h264 @ 0x1e0d960] no frame!
[h264 @ 0x1e0d960] mmco: unref short failure
    Last message repeated 1 times
[h264 @ 0x1e0d960] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
Input #0, h264, from 'stream.lavf.264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 25.25 fps, 25 tbr, 1200k tbn, 50 tbc
Please use -q:a or -q:v, -qscale is ambiguous
Output #0, avi, to 'out1.avi':
  Metadata:
    ISFT            : Lavf55.33.101
    Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 10 tbn, 10 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> mpeg4)
Press [q] to stop, [?] for help
[h264 @ 0x1e60100] Found reference and non-reference fields in the same frame, which is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[h264 @ 0x1e60100] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/MPlayer/incoming/ and contact the ffmpeg-devel mailing list.
[h264 @ 0x1e60100] decode_slice_header error
[h264 @ 0x1e60860] reference picture missing during reorder
[h264 @ 0x1e60860] Missing reference picture, default is 2147483647
[h264 @ 0x1e61720] mmco: unref short failure
[h264 @ 0x1dfd0c0] reference picture missing during reorder
[h264 @ 0x1dfd0c0] Missing reference picture, default is 65300
[h264 @ 0x1e60100] mmco: unref short failure
[h264 @ 0x1dfd0c0] mmco: unref short failure
[h264 @ 0x1e62d40] error while decoding MB 70 42, bytestream -33
frame=   13 fps=0.0 q=2.0 Lsize=    1294kB time=00:00:01.30 bitrate=8154.5kbits/s
video:1288kB audio:0kB subtitle:0 data:0 global headers:0kB muxing overhead 0.461020%
$ ffmpeg -skip_initial_bytes 20 -r 10 -i stream.lavf.264 -qscale 2 out2.avi
ffmpeg version N-61165-g742d860 Copyright (c) 2000-2014 the FFmpeg developers
  built on Mar  8 2014 09:01:38 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      52. 66.101 / 52. 66.101
  libavcodec     55. 52.102 / 55. 52.102
  libavformat    55. 33.101 / 55. 33.101
  libavdevice    55. 11.100 / 55. 11.100
  libavfilter     4.  3.100 /  4.  3.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 18.100 /  0. 18.100
  libpostproc    52.  3.100 / 52.  3.100
[h264 @ 0x254c960] mmco: unref short failure
    Last message repeated 1 times
[h264 @ 0x254c960] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
Input #0, h264, from 'stream.lavf.264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 25.25 fps, 25 tbr, 1200k tbn, 50 tbc
Please use -q:a or -q:v, -qscale is ambiguous
Output #0, avi, to 'out2.avi':
  Metadata:
    ISFT            : Lavf55.33.101
    Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 10 tbn, 10 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> mpeg4)
Press [q] to stop, [?] for help
[h264 @ 0x36f56c0] mmco: unref short failure
    Last message repeated 1 times
[h264 @ 0x36f56c0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0x391db80] error while decoding MB 70 42, bytestream -33
frame=   45 fps=0.0 q=2.0 Lsize=    4006kB time=00:00:04.50 bitrate=7291.9kbits/s
video:3999kB audio:0kB subtitle:0 data:0 global headers:0kB muxing overhead 0.167724%

Skipping the first 20 bytes produces more output frames, adding -flags2 +showall to the first command line shows that the frames are omitted because they are visually broken. The same frames do not look broken if the first 20 bytes are skipped as in the second command line.
I added -r 10 to make the effect more visible in the output files, it is not necessary though.

Using -skip_initial_bytes 20 leads to bit-exact decoding wrt the reference decoder.

Note: See TracTickets for help on using tickets.