Conversion from yuv410p to rgb24 looks wrong

[Jean Delvare]

Summary of the bug:
When generating still pictures from a yuv410p video stream in PNG or BMP format, the output looks incorrect. The same still pictures written as TIFF or JPEG pictures look right. Given that TIFF and JPEG store the data in YUV color space while PNG and BMP use the RGB color space, this makes me suspect that the conversion from yuv410p to rgb24 is broken.

How to reproduce:

% ffmpeg -f lavfi -i "color=color=white:size=16x16" -frames:v 1 -vf "format=yuv410p,drawbox=color=red:x=4:y=4:w=8:h=8" -f image2 410p.png
ffmpeg version N-77180-g45938f0 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.8 (SUSE Linux)
  configuration: --prefix=/opt/ffmpeg --enable-shared --enable-libmp3lame --enable-libxvid --enable-postproc --enable-gpl --enable-x11grab --enable-avfilter --enable-version3 --enable-libx264 --enable-pthreads --enable-libvorbis --enable-libtheora --enable-fontconfig --cpu=corei7 --enable-libfaac --enable-nonfree
  libavutil      55. 10.100 / 55. 10.100
  libavcodec     57. 17.100 / 57. 17.100
  libavformat    57. 19.100 / 57. 19.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6. 20.100 /  6. 20.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, lavfi, from 'color=color=white:size=16x16':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 16x16 [SAR 1:1 DAR 1:1], 25 tbr, 25 tbn, 25 tbc
Output #0, image2, to '410p.png':
  Metadata:
    encoder         : Lavf57.19.100
    Stream #0:0: Video: png, rgb24, 16x16 [SAR 1:1 DAR 1:1], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.17.100 png
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> png (native))
Press [q] to stop, [?] for help
[swscaler @ 0x1b9f4c0] Warning: data is not aligned! This can lead to a speedloss
frame=    1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

The generated image file does not look good. While the generated pattern is fully symmetric, the output image is not. Same problem with 410p.bmp as the output. However both 410p.jpg and 410p.tiff look right.

Also note that format=yuv420p and format=yuv411p both work properly. So this is not a generic problem with the YUV to RGB color space conversion, rather this seems to be specifically related to yuv410p.

Converting from yuv410p to yuv420p seems to also be broken, but differently. See how

% ffmpeg -f lavfi -i "color=color=white:size=16x16" -frames:v 1 -vf "format=yuv410p,drawbox=color=red:x=4:y=4:w=8:h=8,format=yuv420p" -f image2 410p_to_420p.tiff

manages to create green pixels from a picture that originally only had white and red.

[Jean Delvare]

TIFF output from yuv410p source (good)

[Jean Delvare]

yuv410p source to yuv420p (bad)

[Carl Eugen Hoyos]

If there is an issue, it has nothing to do with libavcodec and can be best shown with the following conversion from yuv410p input:

$ ffmpeg -i 410p.tiff out.png
ffmpeg version N-77197-gdf2ce13 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      55. 10.100 / 55. 10.100
  libavcodec     57. 17.100 / 57. 17.100
  libavformat    57. 19.100 / 57. 19.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6. 20.100 /  6. 20.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, tiff_pipe, from '410p.tiff':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: tiff, yuv410p, 16x16 [SAR 1:1 DAR 1:1], 25 tbr, 25 tbn, 25 tbc
Output #0, image2, to 'out.png':
  Metadata:
    encoder         : Lavf57.19.100
    Stream #0:0: Video: png, rgb24, 16x16 [SAR 1:1 DAR 1:1], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.17.100 png
Stream mapping:
  Stream #0:0 -> #0:0 (tiff (native) -> png (native))
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

It seems to me that ImageMagick succeeds with a more bitexact colourspace transformation.

[Jean Delvare]

You are correct that the problem is with swscale.

If I pass -sws_flags neighbor then the output is correct and looks the same as what "convert" produces. So I guess that ImageMagick doesn't perform any upscaling of the chroma planes but simply applies the chroma samples directly to 4x4 areas (which is what I originally expected from ffmpeg.)

Also note that if I change the resolution from 16x16 to 32x32 with the same 8x8 red square at 4,4, the other algorithms, including the default (bicubic) algorithm, produce different results for the common 16x16 part.

[Niklas Haas]

Tentative fix: https://ffmpeg.org//pipermail/ffmpeg-devel/2024-December/337598.html

This eliminates the chroma shift, although there is still a weird bit of extra noise near the bottom row of the image (versus scaling to yuv444p) that appears to be a different bug.

[Niklas Haas]

Tentatively closing this one for now, as at least the completely wrong chroma shift is fixed. The image still does not look quite as high quality as the general scaling path, but I will defer that one to the upcoming rewrite of the core scaling primitives.

[Gyan]

Chroma shift fixed in 095f8038fa9180842cd38d4d61c7c47a02aad9ed