Opened 13 years ago

Closed 13 years ago

#265 closed defect (fixed)

MMX2 yuv420p10 → yuv444p scaler produces vertical undithered(?) lines with GCC 4.6

Reported by: Martin Herkt Owned by: Michael Niedermayer
Priority: normal Component: swscale
Version: git Keywords: yuv420p10, yuv444p, mmx2
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

I know this might be a PITA to reproduce since it only happens with GCC 4.6, but I’ll report it anyway.

Background: I’ve been trying to play yuv420p10 video with my patched MPlayer2 builds for Windows (http://mplayer2.srsfckn.biz/mplayer2-high-bit-depth-latest.7z).
MPlayer2 apparently chooses the yuv420p10 → yuv444p scaler by default.

That is probably fine, but there’s a problem: The MMX2 version of that scaler seems to break with GCC 4.6.0, causing vertical lines of what seems to be undithered luma (can’t tell for sure since I didn’t actually investigate that in detail, but I’ll attach a screenshot).
The C version works fine.

This can be reproduced with for example:
ffmpeg -i test10bit.mkv -pix_fmt yuv444p -vcodec ffv1 out.mkv

Attachments (2)

ffmpeg-dither-h10p.png (96.9 KB ) - added by Martin Herkt 13 years ago.
vertical lines in dithered yuv444p output
swscale-test-results.tar.xz (1.3 MB ) - added by Martin Herkt 13 years ago.
swscale-test results for linux, win32 and win32 without assembly optimization

Download all attachments as: .zip

Change History (7)

by Martin Herkt, 13 years ago

Attachment: ffmpeg-dither-h10p.png added

vertical lines in dithered yuv444p output

comment:1 by Martin Herkt, 13 years ago

Update: It only breaks with MinGW targets (even using GCC 4.5.x), which makes it even more of a PITA to debug. This is probably a MinGW/GCC bug rather than an FFmpeg bug. Sorry if I’m wasting your time here.

Edit: 3DNOW and MMX versions are also affected.

Just a hunch (I have no clue about assembly language at all), but maybe it’s a problem with the memalign hack (which is needed for MinGW)? Even though that only breaking one part of swscale wouldn’t make much sense to me…

Last edited 13 years ago by Martin Herkt (previous) (diff)

comment:2 by Michael Niedermayer, 13 years ago

maybe comparing the output from swscale-test between mingw and linux could help

by Martin Herkt, 13 years ago

Attachment: swscale-test-results.tar.xz added

swscale-test results for linux, win32 and win32 without assembly optimization

in reply to:  2 comment:3 by Michael Niedermayer, 13 years ago

Replying to michael:

maybe comparing the output from swscale-test between mingw and linux could help

which versions did you use to generate these?
also how did you force enable mmx* 3dnow, they are autodetected in recent swscale.

thanks

comment:4 by Roger Pack, 13 years ago

Just ran into this too, with this version

ffplay version git-N-30610-g1929807, built on Jun 7 2011

however this version:

ffplay version N-31706-g335bbe4, built on Jul 31 2011

seems to no longer exhibit the problem for me (it was showing striated lines in mplayer with any software scaler selected, which also appears fixed now with a newer build).

comment:5 by Carl Eugen Hoyos, 13 years ago

Resolution: fixed
Status: newclosed

Thank you for posting, please re-open the issue if still reproducible.

Note: See TracTickets for help on using tickets.