Opened 4 years ago

Last modified 4 years ago

#6815 open defect

Regression recovering broken H264 stream

Reported by: Alex Agranovsky Owned by:
Priority: important Component: avcodec
Version: git-master Keywords: h264 regression
Cc: staircase@sector-alpha.net Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug:

With v3.1.7, the attached H264 input encounters an error, but recovers almost immediately. With v3.3.3 (and current master), it still will recover if AV_CODEC_FLAG2_CHUNKS flag isn't set (which is why ffmpeg works), but will never decode another frame, if the flag is provided.

How to reproduce:

Change ffmpeg.c by adding

ist->dec_ctx->flags2 |= AV_CODEC_FLAG2_CHUNKS;

before avcodec_open2 (not sure if there's a command line option to set it?)

% ffmpeg -threads 1 -i brokenStream.h264 -vcodec mpeg4 foo.mp4

n3.1.7 will recover after encountering an error
concealing 1580 DC, 1580 AC, 1580 MV errors in P frame

n3.3.3 will not

Attachments (2)

brokenStream_cut.h264 (2.4 MB ) - added by Carl Eugen Hoyos 4 years ago.
test.mp4 (907.5 KB ) - added by Alex Agranovsky 4 years ago.
Another instance of broken input

Change History (14)

comment:1 by Carl Eugen Hoyos, 4 years ago

Please provide the input stream.

comment:2 by Alex Agranovsky, 4 years ago

Here goes. For some reason it did not go through the first time.

https://www.dropbox.com/s/g28nnu5vksq1q1r/brokenStream.h264?dl=0

in reply to:  description comment:3 by Carl Eugen Hoyos, 4 years ago

Keywords: h264 regression added
Reproduced by developer: set
Status: newopen
Version: unspecifiedgit-master

Regression since 78c7197ea0e5c53393849a32dd6f49e3b89f7815

$ ffmpeg -flags2 +chunks -threads 1 -i brokenStream_cut.h264 out.mp4
ffmpeg version N-88612-gce52e43 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 6.3.0 (GCC)
  configuration: --enable-gpl
  libavutil      56.  0.100 / 56.  0.100
  libavcodec     58.  1.100 / 58.  1.100
  libavformat    58.  2.100 / 58.  2.100
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  0.101 /  7.  0.101
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, h264, from 'brokenStream_cut.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(progressive), 1280x720, 25 fps, 25 tbr, 1200k tbn, 50 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mpeg4 (native))
Press [q] to stop, [?] for help
Output #0, mp4, to 'out.mp4':
  Metadata:
    encoder         : Lavf58.2.100
    Stream #0:0: Video: mpeg4 (mp4v / 0x7634706D), yuv420p, 1280x720, q=2-31, 200 kb/s, 25 fps, 12800 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.1.100 mpeg4
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
[h264 @ 0x39b7f20] concealing 1580 DC, 1580 AC, 1580 MV errors in P frame
[h264 @ 0x39b7f20] Frame num change from 13 to 14
[h264 @ 0x39b7f20] decode_slice_header error
[h264 @ 0x39b7f20] concealing 1580 DC, 1580 AC, 1580 MV errors in P frame
[h264 @ 0x39b7f20] Frame num change from 13 to 15
[h264 @ 0x39b7f20] decode_slice_header error

...

[h264 @ 0x39b7f20] concealing 1580 DC, 1580 AC, 1580 MV errors in P frame
[h264 @ 0x39b7f20] Frame num change from 13 to 1
[h264 @ 0x39b7f20] decode_slice_header error
[h264 @ 0x39b7f20] concealing 1580 DC, 1580 AC, 1580 MV errors in P frame
[h264 @ 0x39b7f20] Frame num change from 13 to 0
[h264 @ 0x39b7f20] decode_slice_header error
[h264 @ 0x39b7f20] concealing 1580 DC, 1580 AC, 1580 MV errors in P frame
frame=   45 fps=0.0 q=31.0 Lsize=     506kB time=00:00:01.76 bitrate=2356.2kbits/s speed=2.21x
video:505kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.203155%

by Carl Eugen Hoyos, 4 years ago

Attachment: brokenStream_cut.h264 added

comment:4 by Alex Agranovsky, 4 years ago

Hi,

I assume this is a regression in the error recovery, rather than a straight ffmpeg bug. With that assumption, is there a way to explain to the source equipment manufacturer what exactly is broken in the stream they generate, and how they could fix it?

Thanks!

by Alex Agranovsky, 4 years ago

Attachment: test.mp4 added

Another instance of broken input

comment:5 by Simon, 4 years ago

This issue is also causing me problems. The symptoms are exactly the same. In our case the video data is being sent over a network connection that seems to be causing the issues with the stream rather than it being a bad stream generated by the source and as such we have to be as resilient to errors as we possibly can.

Last edited 4 years ago by Simon (previous) (diff)

comment:6 by Simon, 4 years ago

Cc: staircase@sector-alpha.net added

comment:7 by Simon, 4 years ago

H,

Having tested with multiple version while trying to find a build that would work for our code I have narrowed the change down to be between 3.1.11 and 3.2 versions on git and also is in the latest master. I have built using various options and settings using https://github.com/jb-alvarado/media-autobuild_suite.git on windows 7. We for now will use 3.1.11 but this bug effectively blocks us from ever updating to a later version of ffmpeg.

Hope this helps and progress can be made towards a fix for this issue.

Thanks!

comment:8 by Hendrik, 4 years ago

I would strongly recommend to in fact not use the chunked decoding mode, and instead make use of the h264 parser to re-assemble the bitstream. The result should be the same, just using a more thoroughly tested code path.

Last edited 4 years ago by Hendrik (previous) (diff)

in reply to:  8 comment:9 by Simon, 4 years ago

Replying to heleppkes:

I would strongly recommend to in fact not use the chunked decoding mode, and instead make use of the h264 parser to re-assemble the bitstream. The result should be the same, just using a more thoroughly tested code path.

Hi,

I'm not entirely sure how I would go about making use of the h264 parser to re-assemble the bitstream?

Currently I am reading NAL unit packets using av_read_frame and then passing these packets to avcodec_send_packet and pulling the decoded frames out using avcodec_receive_frame. Is this re-assembing done by adding another call to ffmpeg to request it re-assemble the nal-unit packets back into larger nal-unit packets before passing to avcodec_send_packet or an option we pass in when creating the codec_context to request that ffmpeg does it internally?

If you need any more details please can we discuss directly over emails? My email address is "staircase AT sector DASH alpha DOT net".

Thanks,
Simon

Last edited 4 years ago by Simon (previous) (diff)

in reply to:  8 ; comment:10 by Alex Agranovsky, 4 years ago

Replying to heleppkes:

I would strongly recommend to in fact not use the chunked decoding mode, and instead make use of the h264 parser to re-assemble the bitstream. The result should be the same, just using a more thoroughly tested code path.

While I appreciate the recommendation, I'm not confident it applies in the cases where a 3rd party/non-FFMPEG network component is being utilized to receive H264 bitstream, and FFMPEG makes first appearance with the decoder.
If I'm wrong, and there's a way to insert a codec parser between receiver and decoder using public FFMPEG API, I'd be thankful for any pointers on how to accomplish that.

in reply to:  10 ; comment:11 by Hendrik, 4 years ago

Replying to staircase27:

I'm not entirely sure how I would go about making use of the h264 parser to re-assemble the bitstream?

Currently I am reading NAL unit packets using av_read_frame

If you are using avformat through av_read_frame already, then you don't need to do anything. Just don't set the chunk decoding flag.

Replying to w3sip:

If I'm wrong, and there's a way to insert a codec parser between receiver and decoder using public FFMPEG API, I'd be thankful for any pointers on how to accomplish that.

Look into av_parser_parse2 and the related functions in avcodec.h, which are used to do this. If you use an internal demuxer it'll do that automatically, if you use an external one (like you seem to be), its best to use that to create a fully reconstructed bitstream using a parser.

in reply to:  11 comment:12 by Simon, 4 years ago

Replying to heleppkes:

Replying to staircase27:

I'm not entirely sure how I would go about making use of the h264 parser to re-assemble the bitstream?

Currently I am reading NAL unit packets using av_read_frame

If you are using avformat through av_read_frame already, then you don't need to do anything. Just don't set the chunk decoding flag.

Replying to w3sip:

If I'm wrong, and there's a way to insert a codec parser between receiver and decoder using public FFMPEG API, I'd be thankful for any pointers on how to accomplish that.

Look into av_parser_parse2 and the related functions in avcodec.h, which are used to do this. If you use an internal demuxer it'll do that automatically, if you use an external one (like you seem to be), its best to use that to create a fully reconstructed bitstream using a parser.

I am already using av_read_frame but it does still need the CHUNKS decoding flag for some of the streams we work with. Any suggestions for how to fix this? The streams that need the CHUNKS flag use slices rather than fragmentation of large nal-units.

Any suggestions?

Thanks!

Note: See TracTickets for help on using tickets.