Opened 3 years ago

Closed 2 years ago

#9259 closed defect (wontfix)

Vulkan ycbcr Conversion with 10 bit

Reported by: David Oard Owned by:
Priority: normal Component: avcodec
Version: unspecified Keywords: Vulkan
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description (last modified by David Oard)

I am using Vulkan to convert ycbcr AVFrame data to RGB using VK_KHR_sampler_ycbcr_conversion extension. It has been working really well with 8 bit videos Once I try to 10 bit videos the screen is primarily green. The issue is when a AVFrame is decoded from avcodec_receive_frame call the data arrays for each plane is packed into a 16-bit word where the lower 10 bits contain the data. The problem is that Vulkan wants the data in the upper 10 bits in each 16-bit word.

Is there a way to tell the decoder to use the upper 10 bits instead of the lower 10 bits? Or can an enhancement be added to handle which methodology to use?

The a detailed description of the formats I'm using with Vulkan checkout following link. Search for the last instance of VK_FORMAT_G10X6_B10X6_R10X6_3PLANE_420_UNORM_3PACK16.

https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkFormat.html

Thanks or any help or direction!

David

Change History (4)

comment:1 by David Oard, 3 years ago

Description: modified (diff)

comment:2 by Carl Eugen Hoyos, 3 years ago

Keywords: VK_KHR_sampler_ycbcr_conversion VK_FORMAT_G10X6_B10X6_R10X6_3PLANE_420_UNORM_3PACK16 removed
Type: enhancementdefect
Version: 4.3.2unspecified

Please confirm that the issue you see is reproducible with current FFmpeg git head.
Or are you just searching for P016, YUVP16 and GBRP16?

comment:3 by David Oard, 3 years ago

I build the current ffmpeg and tried the program again and it there is no change.
Did more digging and found that it may have to do with endianness with Vulkan or FFmpeg. Most of the videos I decode are prores which uses yuv422p10le (little-endian). Vulkan claims to use host endianness somewhere things must get mixed up by the times it renders (nVidia GPU) seems to be treating 10-bit and 12-bit formats as bid-endian. I don't know how to confirm that. Has this defect been encounters? Is there any kind of workaround?

Here is a ffmpeg ticket that has similar issues back in 2018.
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20180420043010.3709-4-atomnuker@gmail.com/

comment:4 by David Oard, 2 years ago

Resolution: wontfix
Status: newclosed

Vulkan's extension VK_KHR_sampler_ycbcr_conversion doesn't give access to the YUV components to add a multiplier to handle high bit videos (10 or above). I changed my shaders to handle YUV conversion the same way as OpenGL.

color.r = 64.25 * texture(sTexture[0], In.uv.xy).r;
color.g = 64.25 * texture(sTexture[1], In.uv.xy).r;
color.b = 64.25 * texture(sTexture[2], In.uv.xy).r;

Note: See TracTickets for help on using tickets.