Opened 20 months ago
Last modified 20 months ago
#10418 new defect
DirectShow, wrong pixel format
Reported by: | Michael Koch | Owned by: | |
---|---|---|---|
Priority: | normal | Component: | avdevice |
Version: | git-master | Keywords: | dshow |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description (last modified by )
There is a problem with a ASI178MM (monochrome) camera and its DirectShow driver. When I check the options with this command
ffmpeg -list_options 1 -f dshow -i video="ASI178MM Camera (ZWO Design)"
I get this output:
[dshow @ 00000203f5b315c0] pixel_format=rgb8 min s=3096x2080 fps=1.0842e-12 max s=3096x2080 fps=30
[dshow @ 00000203f5b315c0] pixel_format=rgb8 min s=1280x960 fps=1.0842e-12 max s=1280x960 fps=30
[dshow @ 00000203f5b315c0] pixel_format=rgb8 min s=1280x800 fps=1.0842e-12 max s=1280x800 fps=30
... (there are more sizes listed, all with the same pixel format rgb8)
When FFmpeg gets a frame from the camera, the detected pixel format is rgb8 (as defined in FFmpeg: 2 bits red, 3 bits green, 3 bits blue). Of course that's wrong, because it's a monochrome camera and the data is 8-bit gray.
My first idea was that it must be a bug in the DirectShow driver. The data is obviously correct 8-bit gray (I did verify that), but it's incorrectly labeled as "rgb8" instead of "gray".
When you look at the specifications of DirectShow
https://learn.microsoft.com/en-us/windows/win32/directshow/uncompressed-rgb-video-subtypes
you see that there exists no "gray" pixel format, and "RGB8" is a different pixel format described as "RGB, 8 bpp, palettized". It seems an uncompressed 8-bit gray frame must be encoded with "RGB8" pixel format, together with a palette which contains 256 levels of gray.
"RGB8" in DirectShow should become "pal8" or "gray" in FFmpeg, and not "rgb8".
By the way, in line 64 of dshow.c there is a comment that this pixel format is untested. So this might be the first test.
Attachments (1)
Change History (20)
comment:1 by , 20 months ago
Description: | modified (diff) |
---|
comment:2 by , 20 months ago
comment:3 by , 20 months ago
I suspect the problem is in line 70 of dshow.c, but I can't make a test because I can't compile FFmpeg under Windows.
"RGB8" is in the Directshow documentation defined as "RGB, 8 bpp, palettized".
"AV_PIX_FMT_RGB8" in FFmpeg is defined in line 86 of pixfmt.h as "packed RGB 3:3:2, 8bpp, (msb)2R 3G 3B(lsb)".
These are two different pixel formats.
Shouldn't line 70 in dshow.c be changed to AV_PIX_FMT_PAL8, and the palette must be loaded from the DirectShow driver?
The same problem is also in line 68 for RGB4, and in line 66 for RGB1 (here it should work, but the colors from the palette are lost and replaced by black and white).
comment:6 by , 20 months ago
No, it is a dup, because avpriv_find_pix_fmt() does not find RGB formats.
follow-up: 12 comment:7 by , 20 months ago
dshow.c, but I can't make a test because I can't compile FFmpeg under Windows.
Just use github actions pointing to your FFmpeg fork to compile. For that clone https://github.com/BtbN/FFmpeg-Builds enable actions and press build and publish a release. Do not forget to change the repo of FFmpeg.
follow-up: 9 comment:8 by , 20 months ago
It seems an uncompressed 8-bit gray frame must be encoded with "RGB8" pixel format, together with a palette which contains 256 levels of gray.
Where is the code that will decode palette?? How does it look?
comment:9 by , 20 months ago
Replying to Balling:
It seems an uncompressed 8-bit gray frame must be encoded with "RGB8" pixel format, together with a palette which contains 256 levels of gray.
Where is the code that will decode palette?? How does it look?
"For palettized formats, the color of each pixel is specified as an index into a palette. The palette must be included in the format block, following the BITMAPINFOHEADER structure."
Source: https://learn.microsoft.com/en-us/windows/win32/directshow/uncompressed-rgb-video-subtypes
and some more details when you click on BITMAPINFOHEADER.
by , 20 months ago
Attachment: | screenshot_graphstudionext.png added |
---|
Screenshot of GraphStudioNext with the properties of the video renderer.
comment:10 by , 20 months ago
With GraphStudioNext the output is correct 8-bit grayscale. A screenshot with the properties of the video renderer is attached. I think "Extradata" is the palette.
comment:11 by , 20 months ago
So you need to parse
biClrUsed
Specifies the number of color indices in the color table that are actually used by the bitmap. See Remarks for more information.
biClrImportant
Specifies the number of color indices that are considered important for displaying the bitmap. If this value is zero, all colors are important.
follow-up: 13 comment:12 by , 20 months ago
Replying to Balling:
dshow.c, but I can't make a test because I can't compile FFmpeg under Windows.
Just use github actions pointing to your FFmpeg fork to compile. For that clone https://github.com/BtbN/FFmpeg-Builds enable actions and press build and publish a release. Do not forget to change the repo of FFmpeg.
Is this somewhere explained in more detail step by step? I don't understand anything of this github stuff.
comment:13 by , 20 months ago
Replying to Michael Koch:
Replying to Balling:
dshow.c, but I can't make a test because I can't compile FFmpeg under Windows.
Just use github actions pointing to your FFmpeg fork to compile. For that clone https://github.com/BtbN/FFmpeg-Builds enable actions and press build and publish a release. Do not forget to change the repo of FFmpeg.
Is this somewhere explained in more detail step by step? I don't understand anything of this github stuff.
No, that is what I use.
comment:14 by , 20 months ago
A simple bugfix could be in line 70 of dshow.c to replace AV_PIX_FMT_RGB8 by AV_PIX_FMT_GRAY8. I know that it's a hack and the correct way is to use AV_PIX_FMT_PAL8 and read the palette.
But for this camera the hack should work because the palette does anyway contain only 256 shades of gray. Because nobody noticed this issue earlier, that probably means that there exist no color cameras that use the RGB8 pixel format with a non-grayscale palette.
follow-up: 16 comment:15 by , 20 months ago
hack should work
So you did not test it?
With GraphStudioNext the output is correct 8-bit grayscale
Are you sure it does not read palette?
Also can you paste all 1040 bytes of the palette here?
comment:16 by , 20 months ago
Replying to Balling:
hack should work
So you did not test it?
As said before, I can't compile FFmpeg under Windows.
With GraphStudioNext the output is correct 8-bit grayscale
Are you sure it does not read palette?
I'm quite sure that GraphStudioNext does read the palette.
Also can you paste all 1040 bytes of the palette here?
00 00 00 00 01 01 01 00 02 02 02 00 03 03 03 00 04 04 04 00 05 05 05 00 06 06 06 00 07 07 07 00 08 08 08 00 09 09 09 00 0A 0A 0A 00 0B 0B 0B 00 0C 0C 0C 00 0D 0D 0D 00 0E 0E 0E 00 0F 0F 0F 00 10 10 10 00 11 11 11 00 12 12 12 00 13 13 13 00 14 14 14 00 15 15 15 00 16 16 16 00 17 17 17 00 18 18 18 00 19 19 19 00 1A 1A 1A 00 1B 1B 1B 00 1C 1C 1C 00 1D 1D 1D 00 1E 1E 1E 00 1F 1F 1F 00 20 20 20 00 21 21 21 00 22 22 22 00 23 23 23 00 24 24 24 00 25 25 25 00 26 26 26 00 27 27 27 00 28 28 28 00 29 29 29 00 2A 2A 2A 00 2B 2B 2B 00 2C 2C 2C 00 2D 2D 2D 00 2E 2E 2E 00 2F 2F 2F 00 30 30 30 00 31 31 31 00 32 32 32 00 33 33 33 00 34 34 34 00 35 35 35 00 36 36 36 00 37 37 37 00 38 38 38 00 39 39 39 00 3A 3A 3A 00 3B 3B 3B 00 3C 3C 3C 00 3D 3D 3D 00 3E 3E 3E 00 3F 3F 3F 00 40 40 40 00 41 41 41 00 42 42 42 00 43 43 43 00 44 44 44 00 45 45 45 00 46 46 46 00 47 47 47 00 48 48 48 00 49 49 49 00 4A 4A 4A 00 4B 4B 4B 00 4C 4C 4C 00 4D 4D 4D 00 4E 4E 4E 00 4F 4F 4F 00 50 50 50 00 51 51 51 00 52 52 52 00 53 53 53 00 54 54 54 00 55 55 55 00 56 56 56 00 57 57 57 00 58 58 58 00 59 59 59 00 5A 5A 5A 00 5B 5B 5B 00 5C 5C 5C 00 5D 5D 5D 00 5E 5E 5E 00 5F 5F 5F 00 60 60 60 00 61 61 61 00 62 62 62 00 63 63 63 00 64 64 64 00 65 65 65 00 66 66 66 00 67 67 67 00 68 68 68 00 69 69 69 00 6A 6A 6A 00 6B 6B 6B 00 6C 6C 6C 00 6D 6D 6D 00 6E 6E 6E 00 6F 6F 6F 00 70 70 70 00 71 71 71 00 72 72 72 00 73 73 73 00 74 74 74 00 75 75 75 00 76 76 76 00 77 77 77 00 78 78 78 00 79 79 79 00 7A 7A 7A 00 7B 7B 7B 00 7C 7C 7C 00 7D 7D 7D 00 7E 7E 7E 00 7F 7F 7F 00 80 80 80 00 81 81 81 00 82 82 82 00 83 83 83 00 84 84 84 00 85 85 85 00 86 86 86 00 87 87 87 00 88 88 88 00 89 89 89 00 8A 8A 8A 00 8B 8B 8B 00 8C 8C 8C 00 8D 8D 8D 00 8E 8E 8E 00 8F 8F 8F 00 90 90 90 00 91 91 91 00 92 92 92 00 93 93 93 00 94 94 94 00 95 95 95 00 96 96 96 00 97 97 97 00 98 98 98 00 99 99 99 00 9A 9A 9A 00 9B 9B 9B 00 9C 9C 9C 00 9D 9D 9D 00 9E 9E 9E 00 9F 9F 9F 00 A0 A0 A0 00 A1 A1 A1 00 A2 A2 A2 00 A3 A3 A3 00 A4 A4 A4 00 A5 A5 A5 00 A6 A6 A6 00 A7 A7 A7 00 A8 A8 A8 00 A9 A9 A9 00 AA AA AA 00 AB AB AB 00 AC AC AC 00 AD AD AD 00 AE AE AE 00 AF AF AF 00 B0 B0 B0 00 B1 B1 B1 00 B2 B2 B2 00 B3 B3 B3 00 B4 B4 B4 00 B5 B5 B5 00 B6 B6 B6 00 B7 B7 B7 00 B8 B8 B8 00 B9 B9 B9 00 BA BA BA 00 BB BB BB 00 BC BC BC 00 BD BD BD 00 BE BE BE 00 BF BF BF 00 C0 C0 C0 00 C1 C1 C1 00 C2 C2 C2 00 C3 C3 C3 00 C4 C4 C4 00 C5 C5 C5 00 C6 C6 C6 00 C7 C7 C7 00 C8 C8 C8 00 C9 C9 C9 00 CA CA CA 00 CB CB CB 00 CC CC CC 00 CD CD CD 00 CE CE CE 00 CF CF CF 00 D0 D0 D0 00 D1 D1 D1 00 D2 D2 D2 00 D3 D3 D3 00 D4 D4 D4 00 D5 D5 D5 00 D6 D6 D6 00 D7 D7 D7 00 D8 D8 D8 00 D9 D9 D9 00 DA DA DA 00 DB DB DB 00 DC DC DC 00 DD DD DD 00 DE DE DE 00 DF DF DF 00 E0 E0 E0 00 E1 E1 E1 00 E2 E2 E2 00 E3 E3 E3 00 E4 E4 E4 00 E5 E5 E5 00 E6 E6 E6 00 E7 E7 E7 00 E8 E8 E8 00 E9 E9 E9 00 EA EA EA 00 EB EB EB 00 EC EC EC 00 ED ED ED 00 EE EE EE 00 EF EF EF 00 F0 F0 F0 00 F1 F1 F1 00 F2 F2 F2 00 F3 F3 F3 00 F4 F4 F4 00 F5 F5 F5 00 F6 F6 F6 00 F7 F7 F7 00 F8 F8 F8 00 F9 F9 F9 00 FA FA FA 00 FB FB FB 00 FC FC FC 00 FD FD FD 00 FE FE FE 00 FF FF FF 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
follow-up: 19 comment:18 by , 20 months ago
Replying to Balling:
Okay, so palette AF AF AF 00 is R, G, B, A. Hm...
"A" makes no sense for a camera. I think the 4th byte is always 00.
comment:19 by , 20 months ago
Replying to Michael Koch:
Replying to Balling:
Okay, so palette AF AF AF 00 is R, G, B, A. Hm...
"A" makes no sense for a camera. I think the 4th byte is always 00.
It is RGBQUAD, i.e. it has 4 bytes, where 4th is "Alpha or Don't Care".
That is YCbCr, except it has Y plane only.