Opened 3 years ago

Closed 3 years ago

#5249 closed defect (fixed)

VP9 RGB completely broken

Reported by: Cigaes Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: libvpx
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Encoding/decoding VP9 in RGB colorspace is completely broken. I could not find a reference RGB file to find out if the problem lies with the encoder or decoder. A few test cases:

./ffmpeg_g -lavfi testsrc2=d=10 -c libvpx-vp9 -colorspace rgb -y /tmp/test_rgb.webm

[ full console output: E1 ]
→ works (printing "yuv420p"), but Iceweasel (44.0-1) says "Video can't be played because the file is corrupt".

./ffmpeg_g -i /tmp/test_rgb.webm -f null -

[ full console output: D1 ]
→ fails to decode anything.

Now, with an extra option, it works a little better:

./ffmpeg_g -lavfi testsrc2=d=10 -c libvpx-vp9 -colorspace rgb -pix_fmt yuv444p -y /tmp/test_rgb2.webm

[ full console output: E2 ]
→ works, video plays in Iceweasel with chroma subsampling obviously wrong.

./ffmpeg_g -i /tmp/test_rgb2.webm -y /tmp/test_rgb2.nut

[ full console output: D2 ]
→ works, but the colors in the resulting MPEG4/NUT file are wrong.

./ffmpeg_g -i /tmp/test_rgb2.webm -y /tmp/test_rgb_%02d.png

[ trimmed console output: D3 ]
→ spews a boatload of error messages, colors are wrong, images are corrupted after the first few ones.

./ffmpeg_g -threads 1 -i /tmp/test_rgb2.webm -y /tmp/test_rgb_%02d.png

[ full console output: D4 ]
→ works (old threading bug with PNG?) but colors are wrong.

./ffmpeg_g -c libvpx-vp9 -i /tmp/test_rgb2.webm -y /tmp/test_rgb2.nut

[ full console output: D5 ]
→ works, colors are wrong.


E1:

ffmpeg version N-78596-g8fdee3e Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205
  configuration: --enable-shared --disable-static --enable-gpl --enable-libx264 --enable-libopus --enable-libass --enable-libfreetype --enable-libvpx --enable-opengl --assert-level=2
  libavutil      55. 18.100 / 55. 18.100
  libavcodec     57. 24.103 / 57. 24.103
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 34.100 /  6. 34.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
[libvpx-vp9 @ 0x1948200] v1.5.0
Output #0, webm, to '/tmp/test_rgb.webm':
  Metadata:
    encoder         : Lavf57.25.100
    Stream #0:0: Video: vp9 (libvpx-vp9), yuv420p(gbr/unknown/unknown), 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 200 kb/s, 25 fps, 1k tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.24.103 libvpx-vp9
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream mapping:
  testsrc2 -> Stream #0:0 (libvpx-vp9)
Press [q] to stop, [?] for help
frame=  250 fps= 39 q=0.0 Lsize=     179kB time=00:00:10.00 bitrate= 146.4kbits/s speed=1.55x    
video:176kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.231651%

D1:

ffmpeg version N-78596-g8fdee3e Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205
  configuration: --enable-shared --disable-static --enable-gpl --enable-libx264 --enable-libopus --enable-libass --enable-libfreetype --enable-libvpx --enable-opengl --assert-level=2
  libavutil      55. 18.100 / 55. 18.100
  libavcodec     57. 24.103 / 57. 24.103
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 34.100 /  6. 34.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
[vp9 @ 0x1fbd480] RGB not supported in profile 0
[vp9 @ 0x1fbd480] Not all references are available
    Last message repeated 123 times
[matroska,webm @ 0x1fbb580] decoding for stream 0 failed
[matroska,webm @ 0x1fbb580] Could not find codec parameters for stream 0 (Video: vp9 (Profile 0), none(pc, gbr/unknown/unknown), 320x240): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
/tmp/test_rgb.webm: could not find codec parameters
Input #0, matroska,webm, from '/tmp/test_rgb.webm':
  Metadata:
    encoder         : Lavf57.25.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 146 kb/s
    Stream #0:0: Video: vp9 (Profile 0), none(pc, gbr/unknown/unknown), 320x240, SAR 1:1 DAR 4:3, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
[buffer @ 0x1fbdf00] Unable to parse option value "-1" as pixel format
    Last message repeated 1 times
[buffer @ 0x1fbdf00] Error setting option pix_fmt to value -1.
[graph 0 input from stream 0:0 @ 0x1fe8940] Error applying options to the filter.
Error opening filters!

E2:

ffmpeg version N-78596-g8fdee3e Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205
  configuration: --enable-shared --disable-static --enable-gpl --enable-libx264 --enable-libopus --enable-libass --enable-libfreetype --enable-libvpx --enable-opengl --assert-level=2
  libavutil      55. 18.100 / 55. 18.100
  libavcodec     57. 24.103 / 57. 24.103
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 34.100 /  6. 34.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
[libvpx-vp9 @ 0x13bf540] v1.5.0
Output #0, webm, to '/tmp/test_rgb2.webm':
  Metadata:
    encoder         : Lavf57.25.100
    Stream #0:0: Video: vp9 (libvpx-vp9), yuv444p(gbr/unknown/unknown), 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 200 kb/s, 25 fps, 1k tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.24.103 libvpx-vp9
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream mapping:
  testsrc2 -> Stream #0:0 (libvpx-vp9)
Press [q] to stop, [?] for help
frame=  250 fps= 29 q=0.0 Lsize=     228kB time=00:00:10.00 bitrate= 186.8kbits/s speed=1.18x    
video:226kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.962366%

D2:

ffmpeg version N-78596-g8fdee3e Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205
  configuration: --enable-shared --disable-static --enable-gpl --enable-libx264 --enable-libopus --enable-libass --enable-libfreetype --enable-libvpx --enable-opengl --assert-level=2
  libavutil      55. 18.100 / 55. 18.100
  libavcodec     57. 24.103 / 57. 24.103
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 34.100 /  6. 34.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, matroska,webm, from '/tmp/test_rgb2.webm':
  Metadata:
    encoder         : Lavf57.25.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 186 kb/s
    Stream #0:0: Video: vp9 (Profile 1), gbrp(pc, gbr/unknown/unknown), 320x240, SAR 1:1 DAR 4:3, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
Output #0, nut, to '/tmp/test_rgb2.nut':
  Metadata:
    encoder         : Lavf57.25.100
    Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 320x240 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 51200 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.24.103 mpeg4
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (native) -> mpeg4 (native))
Press [q] to stop, [?] for help
frame=  250 fps=0.0 q=10.0 Lsize=     347kB time=00:00:10.00 bitrate= 284.3kbits/s speed=35.7x    
video:346kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.412033%

D3:

ffmpeg version N-78596-g8fdee3e Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205
  configuration: --enable-shared --disable-static --enable-gpl --enable-libx264 --enable-libopus --enable-libass --enable-libfreetype --enable-libvpx --enable-opengl --assert-level=2
  libavutil      55. 18.100 / 55. 18.100
  libavcodec     57. 24.103 / 57. 24.103
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 34.100 /  6. 34.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, matroska,webm, from '/tmp/test_rgb2.webm':
  Metadata:
    encoder         : Lavf57.25.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 186 kb/s
    Stream #0:0: Video: vp9 (Profile 1), gbrp(pc, gbr/unknown/unknown), 320x240, SAR 1:1 DAR 4:3, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
Output #0, image2, to '/tmp/test_rgb_%02d.png':
  Metadata:
    encoder         : Lavf57.25.100
    Stream #0:0: Video: png, rgb24, 320x240 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.24.103 png
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (native) -> png (native))
Press [q] to stop, [?] for help
[vp9 @ 0x220fbe0] Failed to initialize decoder for 320x240 @ 82
[vp9 @ 0x222a600] Not all references are available
[vp9 @ 0x2245020] Not all references are available
[vp9 @ 0x225fa40] Not all references are available
[vp9 @ 0x227a460] Not all references are available
[vp9 @ 0x2294e80] Not all references are available
[vp9 @ 0x21d74a0] Not all references are available
Error while decoding stream #0:0: Cannot allocate memory
Error while decoding stream #0:0: Invalid data found when processing input
[vp9 @ 0x220fbe0] Failed to initialize decoder for 320x240 @ 82
Error while decoding stream #0:0: Invalid data found when processing input
[vp9 @ 0x222a600] Not all references are available
Error while decoding stream #0:0: Invalid data found when processing input
[vp9 @ 0x2245020] Not all references are available
<snip>
Error while decoding stream #0:0: Cannot allocate memory
Error while decoding stream #0:0: Invalid data found when processing input
[vp9 @ 0x220fbe0] Failed to initialize decoder for 320x240 @ 82
<snip>
[vp9 @ 0x2294e80] Not all references are available
[vp9 @ 0x21d74a0] Not all references are available
Error while decoding stream #0:0: Cannot allocate memory
Error while decoding stream #0:0: Invalid data found when processing input
[vp9 @ 0x220fbe0] Failed to initialize decoder for 320x240 @ 82
Error while decoding stream #0:0: Invalid data found when processing input
frame=  249 fps=0.0 q=-0.0 Lsize=N/A time=00:00:09.96 bitrate=N/A dup=192 drop=0 speed=37.4x    
video:19429kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!

D4:

ffmpeg version N-78596-g8fdee3e Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205
  configuration: --enable-shared --disable-static --enable-gpl --enable-libx264 --enable-libopus --enable-libass --enable-libfreetype --enable-libvpx --enable-opengl --assert-level=2
  libavutil      55. 18.100 / 55. 18.100
  libavcodec     57. 24.103 / 57. 24.103
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 34.100 /  6. 34.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, matroska,webm, from '/tmp/test_rgb2.webm':
  Metadata:
    encoder         : Lavf57.25.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 186 kb/s
    Stream #0:0: Video: vp9 (Profile 1), gbrp(pc, gbr/unknown/unknown), 320x240, SAR 1:1 DAR 4:3, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
Output #0, image2, to '/tmp/test_rgb_%02d.png':
  Metadata:
    encoder         : Lavf57.25.100
    Stream #0:0: Video: png, rgb24, 320x240 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.24.103 png
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (native) -> png (native))
Press [q] to stop, [?] for help
frame=  250 fps=0.0 q=-0.0 Lsize=N/A time=00:00:10.00 bitrate=N/A speed=46.3x    
video:7999kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

D5:

ffmpeg version N-78596-g8fdee3e Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205
  configuration: --enable-shared --disable-static --enable-gpl --enable-libx264 --enable-libopus --enable-libass --enable-libfreetype --enable-libvpx --enable-opengl --assert-level=2
  libavutil      55. 18.100 / 55. 18.100
  libavcodec     57. 24.103 / 57. 24.103
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 34.100 /  6. 34.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, matroska,webm, from '/tmp/test_rgb2.webm':
  Metadata:
    encoder         : Lavf57.25.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 186 kb/s
    Stream #0:0: Video: vp9 (Profile 1), gbrp(pc, gbr/unknown/unknown), 320x240, SAR 1:1 DAR 4:3, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
[libvpx-vp9 @ 0x2435880] v1.5.0
Output #0, nut, to '/tmp/test_rgb2.nut':
  Metadata:
    encoder         : Lavf57.25.100
    Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 320x240 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 51200 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.24.103 mpeg4
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (libvpx-vp9) -> mpeg4 (native))
Press [q] to stop, [?] for help
frame=  250 fps=0.0 q=10.0 Lsize=     347kB time=00:00:10.00 bitrate= 284.3kbits/s speed=18.3x    
video:346kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.412033%

Change History (9)

comment:1 Changed 3 years ago by cehoyos

  • Keywords libvpx added
  • Status changed from new to open

Please test this patch with -vcodec gbrp -colorspace rgb, my libvpx is too old:

diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c
index 8992497..91499fd 100644
--- a/libavcodec/libvpxenc.c
+++ b/libavcodec/libvpxenc.c
@@ -295,6 +295,7 @@ static int set_pix_fmt(AVCodecContext *avctx, vpx_codec_caps_t codec_caps,
         *img_fmt = VPX_IMG_FMT_I440;
         return 0;
 #endif
+    case AV_PIX_FMT_GBRP:
     case AV_PIX_FMT_YUV444P:
         enccfg->g_profile = 1;
         *img_fmt = VPX_IMG_FMT_I444;

comment:2 Changed 3 years ago by heleppkes

On top of that patch, set_colorspace should probably automatically use RGB when a RGB pixfmt is used, not only when color_space is set.

comment:3 Changed 3 years ago by Cigaes

The patch works with an extra bit to add GBRP to the list of supported formats. Still, Hendrik is right, there should be automatic switch or at least a consistency check. And the high bit depths are probably missing.

comment:4 follow-up: Changed 3 years ago by cehoyos

Imo, the issue is that no encoder should have a private option colorspace.

Thanks to pkg-config=false I was able to test myself: Where did you add the pix_fmt? I didn't find the list yet...

comment:5 in reply to: ↑ 4 ; follow-up: Changed 3 years ago by heleppkes

Replying to cehoyos:

Imo, the issue is that no encoder should have a private option colorspace.

It doesn't, it uses avctx->colorspace which -colorspace sets.

comment:6 Changed 3 years ago by Cigaes

As Hendrik pointed, it is not a private option, the documentation is slightly misleading.

The list is in vpx.c, I have posted the patch on the mailing-list.

comment:7 in reply to: ↑ 5 Changed 3 years ago by cehoyos

Replying to heleppkes:

Replying to cehoyos:

Imo, the issue is that no encoder should have a private option colorspace.

It doesn't, it uses avctx->colorspace which -colorspace sets.

True, thanks.

comment:8 Changed 3 years ago by cehoyos

  • Reproduced by developer set

comment:9 Changed 3 years ago by cehoyos

  • Resolution set to fixed
  • Status changed from open to closed
Note: See TracTickets for help on using tickets.