Colored haze in mpeg4 decoding (purple, yellow,...)

[te36]

Summary:

I have ca. 3 TB/10 years of free-TV recordings created with (older versions) of Mencoder (aka: ffmpeg). These play back fine with older versions of XBMC/VLC (decoded by ffmpeg/libavcodec / mpeg4), but since XBMC 13.1 and VLC 2.2, ffmpeg produces purple haze that is creeping in from the side of the screen when the video is panning.

How to reproduce:

ffplay purple-haze-bug.avi
(on upload.ffmpeg.org or ticket)

Duration: 00:00:16.63, start: 0.000000, bitrate: 1454 kb/s

Stream #0:0: Video: mpeg4 (Simple Profile) (FMP4 / 0x34504D46), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 1383 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc

...

Latest working versions:

<=ffmpeg 2.2.x (tested 2.2.16, 2.1, 2.0)
ffmpeg

tag 03bb99ae1a99fa315621308b885a8fc70702c9bc Jun 1, 2014 master branch

ffplay version N-63655-g03bb99a Copyright (c) 2003-2014 the FFmpeg developers

built on Nov 9 2016 14:43:54 with gcc 4.9.3 (Gentoo 4.9.3 p1.5, pie-0.6.4)
configuration:
libavutil 52. 88.100 / 52. 88.100
libavcodec 55. 66.100 / 55. 66.100
libavformat 55. 42.100 / 55. 42.100
libavdevice 55. 13.101 / 55. 13.101
libavfilter 4. 5.100 / 4. 5.100
libswscale 2. 6.100 / 2. 6.100
libswresample 0. 19.100 / 0. 19.100

First purple haze version:

=ffmpeg 2.3.0 (tested 2.3 and later).

ffmpeg

tag 9236f7b5a23b4907f7b2bf6346ecd88e6d76f1e0 Jun 14, 2014 master branch

ffplay version N-63957-g9236f7b Copyright (c) 2003-2014 the FFmpeg developers

built on Nov 9 2016 14:50:33 with gcc 4.9.3 (Gentoo 4.9.3 p1.5, pie-0.6.4)
configuration:
libavutil 52. 89.100 / 52. 89.100
libavcodec 55. 66.101 / 55. 66.101
libavformat 55. 43.100 / 55. 43.100
libavdevice 55. 13.101 / 55. 13.101
libavfilter 4. 8.100 / 4. 8.100
libswscale 2. 6.100 / 2. 6.100
libswresample 0. 19.100 / 0. 19.100

I am only _guessing_ this is an avcodec issue because it looks very
much related to mpeg4 decoding of blocks panning in from "offscreen".

git diff 03bb99ae1a99fa315621308b885a8fc70702c9bc 9236f7b5a23b4907f7b2bf6346ecd88e6d76f1e0
produce 700k diff and i can not figure out what a likely cause is ;-(

I am sure there is some problem in mencoder, but i hope that ffmpeg will continue to try to decode as good as possible even potentially buggy videos as it did in the past. Especially videos encoded with (older versions of) ffmpeg.

The example video is encoded with MEncoder 1.0rc1-4.1.2, but the problem exists with at least up to Mencoder-1.1(which uses 0.10.2.git).

Sorry for reporting so late, did not have the time earlier and was hoping somebody else would find this too.

[Carl Eugen Hoyos]

Regression since c5fc8ae12622a507d7b9ee30ddcd3734e6de6b1d

[te36]

Andy Furniss suggested that c5fc8ae12622a507d7b9ee30ddcd3734e6de6b1d introduced the problem. I verified that it does:

Applying c5fc8ae12622a507d7b9ee30ddcd3734e6de6b1d to a working older version
of ffmpeg introduces the purple haze.

Reverting c5fc8ae12622a507d7b9ee30ddcd3734e6de6b1d even in latest 3.2 makes
purple haze go away.

[Carl Eugen Hoyos]

Didn't I suggest this?

[Carl Eugen Hoyos]

If I understand the sent patches correctly, the reason for this bug is an interlaced encoding regression since 60ab6e24574a984655800d1f7ce16c05f4e9b28c (?) or a nearby commit.

[te36]

Replying to cehoyos:

Didn't I suggest this?

Sorry, i was referring to Andy's mail on ffmpeg-users.

[te36]

Looking at: ​https://github.com/FFmpeg/FFmpeg/commits/master/libavcodec/mpegvideo_motion.c

It seems that every second commit in 2016/2015 changes the code back and force between the "good old" code (no purple haze) - eg: with variable "uvbuf" and the "new" code - eg: with variable ubuf/vbuf. There is also 39a2d3288e82e4e576c03efb32179ef5a19fff50 that pulled out the code into a separate function (using the good old code), but that fix also vanished later on. Without an explanation why (that i could find).

[Carl Eugen Hoyos]

Please do not duplicate every message here and on -users.

Consider reading ​http://blog.pkh.me/p/13-the-ffmpeg-libav-situation.html to understand why merges were necessary once upon a time.

[Carl Eugen Hoyos]

Decoder- and encoder-bug fixed by Michael in 85407c7e63722a2d723257e8cf5f281a8c9f34a4
A work-around for broken encodings was applied as 2c9106257ffca8faef367a410c16bd8220942f6e

[te36]

Thank you for the quick fix!. Seems to be solving the issues.

Suggested release-note/info for the bugfix:

Certain encoder combinations (eg: mencoder/libxvid) can result in correct mpeg 4 part 2 encodings that also display correctly with non-ffmpeg based decoders (eg: windows media player) but that causes colored haze with ffmpeg decoding based application (eg: VLC, Kodi). This bugfix solves this decoding issue.

Certain versions of FFMpeg caused incorrect mpeg 4 part 2 encodings that cause colored haze with ffmpeg and non-ffmpeg decoders. When decoding those encodings with a fixed version of ffmpeg, it recognizes the incorrect encoding by the FFmpeg version number encoded in the metadata of the media and activate a workaround for the decoding. This workaround can be manually invoked ("bug iedge"). The affected released versions of FFmpeg use libavcodec [55.67.101... 57.64.100] or [57.65.100...57.66.104].

---
Tested:

Checked video archive for encodings with buggy ffmpeg encoding. Luckily only one file.

NOK file does display incorrectly with windows media player (non ffmpeg) and non-fixed ffmpeg. It does display correctly with fixed ffmpeg.

OK files display correctly with windows media player and do play incorrectly with non-fixed ffmpeg/ffplay. They do play correctly with fixed ffmpeg/ffplay.

Tried to re-create OK encodings that cause this problem, but can only recreate with mencoder/libxvid. Using ffmpeg/libxvid with same parameters does not cause problem at all. Maybe i was doing something wrong, if not then encoding problems might have been rare to ffmpeg users.

[Carl Eugen Hoyos]

Note that this issue is not related to xvid, the sample you uploaded was encoded with libavcodec.

[te36]

Oops. Not sure why i got confused into thinking there was libxvidcore used by ffmpeg in my encodings. Just went back and recompiled mencoder totally without libxvidcore and its indeed not used.

[te36]

To reproduce "OK" encoding with the haze problem.

Re-encode example clip with fixed ffmpeg version, eg: to mpeg2.

Re-encode with mencoder 1.2.1:
mencoder-1.2.1 -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=1500:ildct:ilme -o purple-haze-bug-mencoder.avi purple-haze-bug.ts

produces the same large haze areas. This version of mencoder uses Lavc54.23.100

Re-encode instead directly with Lavc54.23.100:

ffmpeg-lavc-54.23.100 -i purple-haze-bug.ts -vcodec mpeg4 -flags +ildct+ilme -b:v 1500k purple-haze-bug-ffmpeg.avi

produces some small amount of haze on the edges but they don't tear across the whole picture. Maybe mencoder feeds some other parameters into lavc mpeg4 encoding.

Recent versions of ffmpeg/mencoder do not produce visible haze anymore using the same encoding example.