Occasional Access Violation: H264 with bad streams

[Warren Black]

Summary of the bug:
When playing a certain unreliable live stream, experienced core-dumps about every four hours on average.
How to reproduce:

ffplay rtmp://planeta-online.tv:1936/live/channel_22

Version: Git trunk sources as of 5/24/13. "version.sh" reports "N-53488-g953e335".
Built: 6/20/13.
Details:
When playing unreliable H264 streams with FFPlay, I seem to get core-dumps randomly every few hours. The exact location is usually the second instruction of "pred8x8_top_dc_8_mmxext" in "h264_intrapred.asm", where it dereferences "dest_cr" after subtracting "uvlinesize" from it, as called from the line reading

h->hpc.pred8x8[h->chroma_pred_mode](dest_cr, uvlinesize);

in "h264_mb_template.c". "uvlinesize" is typically something like 320 at the time of crash, with "mb_y" zero.

My take on this is that, when presented with garbaged stream data, the H264 frame decoder sometimes tries to perform predictions that involve higher rows (lower memory addresses): if "mb_y" happens to be zero (the top row), this means that it tries to read memory from "negative rows", addresses a few hundred bytes before the beginning of the legitimate frame data. Often, those addresses point to harmless random bytes, but occasionally it actually points to unmapped memory pages, causing Access Violations.

[Warren Black]

Possible patch:

diff --git a/libavcodec/h264_mb_template.c b/libavcodec/h264_mb_template.c
index 15cb3c9..0173334 100644
--- a/libavcodec/h264_mb_template.c
+++ b/libavcodec/h264_mb_template.c
@@ -158,8 +158,17 @@ static av_noinline void FUNC(hl_decode_mb)(H264Context *h)
                                uvlinesize, 1, 0, SIMPLE, PIXEL_SHIFT);
 
             if (SIMPLE || !CONFIG_GRAY || !(h->flags & CODEC_FLAG_GRAY)) {
-                h->hpc.pred8x8[h->chroma_pred_mode](dest_cb, uvlinesize);
-                h->hpc.pred8x8[h->chroma_pred_mode](dest_cr, uvlinesize);
+                if (mb_y <= 0 &&
+                    (
+                        h->chroma_pred_mode == TOP_DC_PRED8x8 ||
+                        h->chroma_pred_mode == DC_PRED8x8 ||
+                        h->chroma_pred_mode == VERT_PRED8x8)
+                    ) {
+                    av_log(NULL, AV_LOG_WARNING, "Skipping prediction involving previous data rows because mb_y is zero\n");
+                } else {
+                    h->hpc.pred8x8[h->chroma_pred_mode](dest_cb, uvlinesize);
+                    h->hpc.pred8x8[h->chroma_pred_mode](dest_cr, uvlinesize);
+                }
             }
 
             hl_decode_mb_predict_luma(h, mb_type, is_h264, SIMPLE,

[Michael Niedermayer]

Do you have a small input file with which this can be reproduced ?
listening for hours to that stream is not very practical to debug this

or if you like to debug it yourself
ff_h264_check_intra_pred_mode() should be checking that the mode is valid, there should be no need to check whe the pred function is called

[Warren Black]

I believe this was fixed in commits d6a33f5d20b6ef2eae2cbb959b001cb125a564b7 and 2005fddcbb4e18e8f7c34326e40609e4a2d83c31 on June 21, 2013, thanks. Besides no longer causing access violations, I believe this has reduced the frequency/duration of "green screens" when playing flaky H264 live streams.

[Michael Niedermayer]

ok, closed then.
If it turns out that it hasnt been fixed, then dont hesitate to reopen this ticket

[Carl Eugen Hoyos]

Replying to BlackWarren:

I believe this was fixed in commits d6a33f5d20b6ef2eae2cbb959b001cb125a564b7 and 2005fddcbb4e18e8f7c34326e40609e4a2d83c31 on June 21, 2013, thanks.

I was unable to reproduce the crash with older versions.

Besides no longer causing access violations, I believe this has reduced the frequency/duration of "green screens" when playing flaky H264 live streams.

Such a sample would also be very welcome!

[Warren Black]

The actual crashing (access violation) may be Windows-specific -- under Linux, there may always be a couple of kilobytes of dereferenceable memory right before anything that's in the heap. But still, without the patch, garbage data will be fetched when stream data is unreliable.

The crashing was rare, and hard to reproduce, requiring on the order of 1 GB of streaming data to invoke it. I never bothered to capture a data set that would cause it. I've just observed that green screening during stream play was common before your fix, and now very uncommon.

[Carl Eugen Hoyos]

Replying to BlackWarren:

The actual crashing (access violation) may be Windows-specific -- under Linux, there may always be a couple of kilobytes of dereferenceable memory right before anything that's in the heap.

That's why I tested both on Windows and with valgrind (>1G iirc).

I've just observed that green screening during stream play was common before your fix, and now very uncommon.

If it still happens, please try to record a sample.

[Svetlana]

Replying to cehoyos:

While receiving from local network and decoding 16 streams with 1920x1080 resolution, this happens after ~ 10 minutes of test.
I have such video sample, but I think it doesn't depends on video. It's better to add BlackWarren's patch to source code.

[Carl Eugen Hoyos]

Please do not reopen very old tickets, instead open a new one. Don't forget to test current FFmpeg git head and to provide all necessary information including a sample that allows to reproduce the issue.

[Svetlana]

Replying to cehoyos:
This issue is present in last version of ffmpeg (release 2.8.6).
How can I send you my sample file and test description?

[Carl Eugen Hoyos]

Please test current FFmpeg git head and please read ​http://ffmpeg.org/bugreports.html (there is no file size limit).