Opened 6 months ago

Last modified 6 weeks ago

#6815 open defect

Regression recovering broken H264 stream

Reported by: w3sip Owned by:
Priority: important Component: avcodec
Version: git-master Keywords: h264 regression
Cc: staircase@sector-alpha.net Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug:

With v3.1.7, the attached H264 input encounters an error, but recovers almost immediately. With v3.3.3 (and current master), it still will recover if AV_CODEC_FLAG2_CHUNKS flag isn't set (which is why ffmpeg works), but will never decode another frame, if the flag is provided.

How to reproduce:

Change ffmpeg.c by adding

ist->dec_ctx->flags2 |= AV_CODEC_FLAG2_CHUNKS;

before avcodec_open2 (not sure if there's a command line option to set it?)

% ffmpeg -threads 1 -i brokenStream.h264 -vcodec mpeg4 foo.mp4

n3.1.7 will recover after encountering an error
concealing 1580 DC, 1580 AC, 1580 MV errors in P frame

n3.3.3 will not

Attachments (2)

brokenStream_cut.h264 (2.4 MB) - added by cehoyos 6 months ago.
test.mp4 (907.5 KB) - added by w3sip 5 months ago.
Another instance of broken input

Change History (14)

comment:1 Changed 6 months ago by cehoyos

Please provide the input stream.

comment:2 Changed 6 months ago by w3sip

Here goes. For some reason it did not go through the first time.

https://www.dropbox.com/s/g28nnu5vksq1q1r/brokenStream.h264?dl=0

comment:3 in reply to: ↑ description Changed 6 months ago by cehoyos

  • Keywords h264 regression added
  • Reproduced by developer set
  • Status changed from new to open
  • Version changed from unspecified to git-master

Regression since 78c7197ea0e5c53393849a32dd6f49e3b89f7815

$ ffmpeg -flags2 +chunks -threads 1 -i brokenStream_cut.h264 out.mp4
ffmpeg version N-88612-gce52e43 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 6.3.0 (GCC)
  configuration: --enable-gpl
  libavutil      56.  0.100 / 56.  0.100
  libavcodec     58.  1.100 / 58.  1.100
  libavformat    58.  2.100 / 58.  2.100
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  0.101 /  7.  0.101
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, h264, from 'brokenStream_cut.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(progressive), 1280x720, 25 fps, 25 tbr, 1200k tbn, 50 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mpeg4 (native))
Press [q] to stop, [?] for help
Output #0, mp4, to 'out.mp4':
  Metadata:
    encoder         : Lavf58.2.100
    Stream #0:0: Video: mpeg4 (mp4v / 0x7634706D), yuv420p, 1280x720, q=2-31, 200 kb/s, 25 fps, 12800 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.1.100 mpeg4
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
[h264 @ 0x39b7f20] concealing 1580 DC, 1580 AC, 1580 MV errors in P frame
[h264 @ 0x39b7f20] Frame num change from 13 to 14
[h264 @ 0x39b7f20] decode_slice_header error
[h264 @ 0x39b7f20] concealing 1580 DC, 1580 AC, 1580 MV errors in P frame
[h264 @ 0x39b7f20] Frame num change from 13 to 15
[h264 @ 0x39b7f20] decode_slice_header error

...

[h264 @ 0x39b7f20] concealing 1580 DC, 1580 AC, 1580 MV errors in P frame
[h264 @ 0x39b7f20] Frame num change from 13 to 1
[h264 @ 0x39b7f20] decode_slice_header error
[h264 @ 0x39b7f20] concealing 1580 DC, 1580 AC, 1580 MV errors in P frame
[h264 @ 0x39b7f20] Frame num change from 13 to 0
[h264 @ 0x39b7f20] decode_slice_header error
[h264 @ 0x39b7f20] concealing 1580 DC, 1580 AC, 1580 MV errors in P frame
frame=   45 fps=0.0 q=31.0 Lsize=     506kB time=00:00:01.76 bitrate=2356.2kbits/s speed=2.21x
video:505kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.203155%

Changed 6 months ago by cehoyos

comment:4 Changed 5 months ago by w3sip

Hi,

I assume this is a regression in the error recovery, rather than a straight ffmpeg bug. With that assumption, is there a way to explain to the source equipment manufacturer what exactly is broken in the stream they generate, and how they could fix it?

Thanks!

Changed 5 months ago by w3sip

Another instance of broken input

comment:5 Changed 8 weeks ago by staircase27

This issue is also causing me problems. The symptoms are exactly the same. In our case the video data is being sent over a network connection that seems to be causing the issues with the stream rather than it being a bad stream generated by the source and as such we have to be as resilient to errors as we possibly can.

Last edited 7 weeks ago by staircase27 (previous) (diff)

comment:6 Changed 8 weeks ago by staircase27

  • Cc staircase@sector-alpha.net added

comment:7 Changed 6 weeks ago by staircase27

H,

Having tested with multiple version while trying to find a build that would work for our code I have narrowed the change down to be between 3.1.11 and 3.2 versions on git and also is in the latest master. I have built using various options and settings using https://github.com/jb-alvarado/media-autobuild_suite.git on windows 7. We for now will use 3.1.11 but this bug effectively blocks us from ever updating to a later version of ffmpeg.

Hope this helps and progress can be made towards a fix for this issue.

Thanks!

comment:8 follow-ups: Changed 6 weeks ago by heleppkes

I would strongly recommend to in fact not use the chunked decoding mode, and instead make use of the h264 parser to re-assemble the bitstream. The result should be the same, just using a more thoroughly tested code path.

Last edited 6 weeks ago by heleppkes (previous) (diff)

comment:9 in reply to: ↑ 8 Changed 6 weeks ago by staircase27

Replying to heleppkes:

I would strongly recommend to in fact not use the chunked decoding mode, and instead make use of the h264 parser to re-assemble the bitstream. The result should be the same, just using a more thoroughly tested code path.

Hi,

I'm not entirely sure how I would go about making use of the h264 parser to re-assemble the bitstream?

Currently I am reading NAL unit packets using av_read_frame and then passing these packets to avcodec_send_packet and pulling the decoded frames out using avcodec_receive_frame. Is this re-assembing done by adding another call to ffmpeg to request it re-assemble the nal-unit packets back into larger nal-unit packets before passing to avcodec_send_packet or an option we pass in when creating the codec_context to request that ffmpeg does it internally?

If you need any more details please can we discuss directly over emails? My email address is "staircase AT sector DASH alpha DOT net".

Thanks,
Simon

Last edited 6 weeks ago by staircase27 (previous) (diff)

comment:10 in reply to: ↑ 8 ; follow-up: Changed 6 weeks ago by w3sip

Replying to heleppkes:

I would strongly recommend to in fact not use the chunked decoding mode, and instead make use of the h264 parser to re-assemble the bitstream. The result should be the same, just using a more thoroughly tested code path.

While I appreciate the recommendation, I'm not confident it applies in the cases where a 3rd party/non-FFMPEG network component is being utilized to receive H264 bitstream, and FFMPEG makes first appearance with the decoder.
If I'm wrong, and there's a way to insert a codec parser between receiver and decoder using public FFMPEG API, I'd be thankful for any pointers on how to accomplish that.

comment:11 in reply to: ↑ 10 ; follow-up: Changed 6 weeks ago by heleppkes

Replying to staircase27:

I'm not entirely sure how I would go about making use of the h264 parser to re-assemble the bitstream?

Currently I am reading NAL unit packets using av_read_frame

If you are using avformat through av_read_frame already, then you don't need to do anything. Just don't set the chunk decoding flag.

Replying to w3sip:

If I'm wrong, and there's a way to insert a codec parser between receiver and decoder using public FFMPEG API, I'd be thankful for any pointers on how to accomplish that.

Look into av_parser_parse2 and the related functions in avcodec.h, which are used to do this. If you use an internal demuxer it'll do that automatically, if you use an external one (like you seem to be), its best to use that to create a fully reconstructed bitstream using a parser.

comment:12 in reply to: ↑ 11 Changed 6 weeks ago by staircase27

Replying to heleppkes:

Replying to staircase27:

I'm not entirely sure how I would go about making use of the h264 parser to re-assemble the bitstream?

Currently I am reading NAL unit packets using av_read_frame

If you are using avformat through av_read_frame already, then you don't need to do anything. Just don't set the chunk decoding flag.

Replying to w3sip:

If I'm wrong, and there's a way to insert a codec parser between receiver and decoder using public FFMPEG API, I'd be thankful for any pointers on how to accomplish that.

Look into av_parser_parse2 and the related functions in avcodec.h, which are used to do this. If you use an internal demuxer it'll do that automatically, if you use an external one (like you seem to be), its best to use that to create a fully reconstructed bitstream using a parser.

I am already using av_read_frame but it does still need the CHUNKS decoding flag for some of the streams we work with. Any suggestions for how to fix this? The streams that need the CHUNKS flag use slices rather than fragmentation of large nal-units.

Any suggestions?

Thanks!

Note: See TracTickets for help on using tickets.