Opened 8 years ago

Closed 8 years ago

#5249 closed defect (fixed)

VP9 RGB completely broken

Reported by: Cigaes Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: libvpx
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Encoding/decoding VP9 in RGB colorspace is completely broken. I could not find a reference RGB file to find out if the problem lies with the encoder or decoder. A few test cases:

./ffmpeg_g -lavfi testsrc2=d=10 -c libvpx-vp9 -colorspace rgb -y /tmp/test_rgb.webm

[ full console output: E1 ]
→ works (printing "yuv420p"), but Iceweasel (44.0-1) says "Video can't be played because the file is corrupt".

./ffmpeg_g -i /tmp/test_rgb.webm -f null -

[ full console output: D1 ]
→ fails to decode anything.

Now, with an extra option, it works a little better:

./ffmpeg_g -lavfi testsrc2=d=10 -c libvpx-vp9 -colorspace rgb -pix_fmt yuv444p -y /tmp/test_rgb2.webm

[ full console output: E2 ]
→ works, video plays in Iceweasel with chroma subsampling obviously wrong.

./ffmpeg_g -i /tmp/test_rgb2.webm -y /tmp/test_rgb2.nut

[ full console output: D2 ]
→ works, but the colors in the resulting MPEG4/NUT file are wrong.

./ffmpeg_g -i /tmp/test_rgb2.webm -y /tmp/test_rgb_%02d.png

[ trimmed console output: D3 ]
→ spews a boatload of error messages, colors are wrong, images are corrupted after the first few ones.

./ffmpeg_g -threads 1 -i /tmp/test_rgb2.webm -y /tmp/test_rgb_%02d.png

[ full console output: D4 ]
→ works (old threading bug with PNG?) but colors are wrong.

./ffmpeg_g -c libvpx-vp9 -i /tmp/test_rgb2.webm -y /tmp/test_rgb2.nut

[ full console output: D5 ]
→ works, colors are wrong.


E1:

ffmpeg version N-78596-g8fdee3e Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205
  configuration: --enable-shared --disable-static --enable-gpl --enable-libx264 --enable-libopus --enable-libass --enable-libfreetype --enable-libvpx --enable-opengl --assert-level=2
  libavutil      55. 18.100 / 55. 18.100
  libavcodec     57. 24.103 / 57. 24.103
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 34.100 /  6. 34.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
[libvpx-vp9 @ 0x1948200] v1.5.0
Output #0, webm, to '/tmp/test_rgb.webm':
  Metadata:
    encoder         : Lavf57.25.100
    Stream #0:0: Video: vp9 (libvpx-vp9), yuv420p(gbr/unknown/unknown), 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 200 kb/s, 25 fps, 1k tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.24.103 libvpx-vp9
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream mapping:
  testsrc2 -> Stream #0:0 (libvpx-vp9)
Press [q] to stop, [?] for help
frame=  250 fps= 39 q=0.0 Lsize=     179kB time=00:00:10.00 bitrate= 146.4kbits/s speed=1.55x    
video:176kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.231651%

D1:

ffmpeg version N-78596-g8fdee3e Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205
  configuration: --enable-shared --disable-static --enable-gpl --enable-libx264 --enable-libopus --enable-libass --enable-libfreetype --enable-libvpx --enable-opengl --assert-level=2
  libavutil      55. 18.100 / 55. 18.100
  libavcodec     57. 24.103 / 57. 24.103
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 34.100 /  6. 34.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
[vp9 @ 0x1fbd480] RGB not supported in profile 0
[vp9 @ 0x1fbd480] Not all references are available
    Last message repeated 123 times
[matroska,webm @ 0x1fbb580] decoding for stream 0 failed
[matroska,webm @ 0x1fbb580] Could not find codec parameters for stream 0 (Video: vp9 (Profile 0), none(pc, gbr/unknown/unknown), 320x240): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
/tmp/test_rgb.webm: could not find codec parameters
Input #0, matroska,webm, from '/tmp/test_rgb.webm':
  Metadata:
    encoder         : Lavf57.25.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 146 kb/s
    Stream #0:0: Video: vp9 (Profile 0), none(pc, gbr/unknown/unknown), 320x240, SAR 1:1 DAR 4:3, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
[buffer @ 0x1fbdf00] Unable to parse option value "-1" as pixel format
    Last message repeated 1 times
[buffer @ 0x1fbdf00] Error setting option pix_fmt to value -1.
[graph 0 input from stream 0:0 @ 0x1fe8940] Error applying options to the filter.
Error opening filters!

E2:

ffmpeg version N-78596-g8fdee3e Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205
  configuration: --enable-shared --disable-static --enable-gpl --enable-libx264 --enable-libopus --enable-libass --enable-libfreetype --enable-libvpx --enable-opengl --assert-level=2
  libavutil      55. 18.100 / 55. 18.100
  libavcodec     57. 24.103 / 57. 24.103
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 34.100 /  6. 34.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
[libvpx-vp9 @ 0x13bf540] v1.5.0
Output #0, webm, to '/tmp/test_rgb2.webm':
  Metadata:
    encoder         : Lavf57.25.100
    Stream #0:0: Video: vp9 (libvpx-vp9), yuv444p(gbr/unknown/unknown), 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 200 kb/s, 25 fps, 1k tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.24.103 libvpx-vp9
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream mapping:
  testsrc2 -> Stream #0:0 (libvpx-vp9)
Press [q] to stop, [?] for help
frame=  250 fps= 29 q=0.0 Lsize=     228kB time=00:00:10.00 bitrate= 186.8kbits/s speed=1.18x    
video:226kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.962366%

D2:

ffmpeg version N-78596-g8fdee3e Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205
  configuration: --enable-shared --disable-static --enable-gpl --enable-libx264 --enable-libopus --enable-libass --enable-libfreetype --enable-libvpx --enable-opengl --assert-level=2
  libavutil      55. 18.100 / 55. 18.100
  libavcodec     57. 24.103 / 57. 24.103
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 34.100 /  6. 34.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, matroska,webm, from '/tmp/test_rgb2.webm':
  Metadata:
    encoder         : Lavf57.25.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 186 kb/s
    Stream #0:0: Video: vp9 (Profile 1), gbrp(pc, gbr/unknown/unknown), 320x240, SAR 1:1 DAR 4:3, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
Output #0, nut, to '/tmp/test_rgb2.nut':
  Metadata:
    encoder         : Lavf57.25.100
    Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 320x240 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 51200 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.24.103 mpeg4
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (native) -> mpeg4 (native))
Press [q] to stop, [?] for help
frame=  250 fps=0.0 q=10.0 Lsize=     347kB time=00:00:10.00 bitrate= 284.3kbits/s speed=35.7x    
video:346kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.412033%

D3:

ffmpeg version N-78596-g8fdee3e Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205
  configuration: --enable-shared --disable-static --enable-gpl --enable-libx264 --enable-libopus --enable-libass --enable-libfreetype --enable-libvpx --enable-opengl --assert-level=2
  libavutil      55. 18.100 / 55. 18.100
  libavcodec     57. 24.103 / 57. 24.103
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 34.100 /  6. 34.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, matroska,webm, from '/tmp/test_rgb2.webm':
  Metadata:
    encoder         : Lavf57.25.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 186 kb/s
    Stream #0:0: Video: vp9 (Profile 1), gbrp(pc, gbr/unknown/unknown), 320x240, SAR 1:1 DAR 4:3, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
Output #0, image2, to '/tmp/test_rgb_%02d.png':
  Metadata:
    encoder         : Lavf57.25.100
    Stream #0:0: Video: png, rgb24, 320x240 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.24.103 png
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (native) -> png (native))
Press [q] to stop, [?] for help
[vp9 @ 0x220fbe0] Failed to initialize decoder for 320x240 @ 82
[vp9 @ 0x222a600] Not all references are available
[vp9 @ 0x2245020] Not all references are available
[vp9 @ 0x225fa40] Not all references are available
[vp9 @ 0x227a460] Not all references are available
[vp9 @ 0x2294e80] Not all references are available
[vp9 @ 0x21d74a0] Not all references are available
Error while decoding stream #0:0: Cannot allocate memory
Error while decoding stream #0:0: Invalid data found when processing input
[vp9 @ 0x220fbe0] Failed to initialize decoder for 320x240 @ 82
Error while decoding stream #0:0: Invalid data found when processing input
[vp9 @ 0x222a600] Not all references are available
Error while decoding stream #0:0: Invalid data found when processing input
[vp9 @ 0x2245020] Not all references are available
<snip>
Error while decoding stream #0:0: Cannot allocate memory
Error while decoding stream #0:0: Invalid data found when processing input
[vp9 @ 0x220fbe0] Failed to initialize decoder for 320x240 @ 82
<snip>
[vp9 @ 0x2294e80] Not all references are available
[vp9 @ 0x21d74a0] Not all references are available
Error while decoding stream #0:0: Cannot allocate memory
Error while decoding stream #0:0: Invalid data found when processing input
[vp9 @ 0x220fbe0] Failed to initialize decoder for 320x240 @ 82
Error while decoding stream #0:0: Invalid data found when processing input
frame=  249 fps=0.0 q=-0.0 Lsize=N/A time=00:00:09.96 bitrate=N/A dup=192 drop=0 speed=37.4x    
video:19429kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!

D4:

ffmpeg version N-78596-g8fdee3e Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205
  configuration: --enable-shared --disable-static --enable-gpl --enable-libx264 --enable-libopus --enable-libass --enable-libfreetype --enable-libvpx --enable-opengl --assert-level=2
  libavutil      55. 18.100 / 55. 18.100
  libavcodec     57. 24.103 / 57. 24.103
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 34.100 /  6. 34.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, matroska,webm, from '/tmp/test_rgb2.webm':
  Metadata:
    encoder         : Lavf57.25.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 186 kb/s
    Stream #0:0: Video: vp9 (Profile 1), gbrp(pc, gbr/unknown/unknown), 320x240, SAR 1:1 DAR 4:3, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
Output #0, image2, to '/tmp/test_rgb_%02d.png':
  Metadata:
    encoder         : Lavf57.25.100
    Stream #0:0: Video: png, rgb24, 320x240 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.24.103 png
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (native) -> png (native))
Press [q] to stop, [?] for help
frame=  250 fps=0.0 q=-0.0 Lsize=N/A time=00:00:10.00 bitrate=N/A speed=46.3x    
video:7999kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

D5:

ffmpeg version N-78596-g8fdee3e Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205
  configuration: --enable-shared --disable-static --enable-gpl --enable-libx264 --enable-libopus --enable-libass --enable-libfreetype --enable-libvpx --enable-opengl --assert-level=2
  libavutil      55. 18.100 / 55. 18.100
  libavcodec     57. 24.103 / 57. 24.103
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 34.100 /  6. 34.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, matroska,webm, from '/tmp/test_rgb2.webm':
  Metadata:
    encoder         : Lavf57.25.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 186 kb/s
    Stream #0:0: Video: vp9 (Profile 1), gbrp(pc, gbr/unknown/unknown), 320x240, SAR 1:1 DAR 4:3, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
[libvpx-vp9 @ 0x2435880] v1.5.0
Output #0, nut, to '/tmp/test_rgb2.nut':
  Metadata:
    encoder         : Lavf57.25.100
    Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 320x240 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 51200 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.24.103 mpeg4
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (libvpx-vp9) -> mpeg4 (native))
Press [q] to stop, [?] for help
frame=  250 fps=0.0 q=10.0 Lsize=     347kB time=00:00:10.00 bitrate= 284.3kbits/s speed=18.3x    
video:346kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.412033%

Change History (9)

comment:1 by Carl Eugen Hoyos, 8 years ago

Keywords: libvpx added
Status: newopen

Please test this patch with -vcodec gbrp -colorspace rgb, my libvpx is too old:

diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c
index 8992497..91499fd 100644
--- a/libavcodec/libvpxenc.c
+++ b/libavcodec/libvpxenc.c
@@ -295,6 +295,7 @@ static int set_pix_fmt(AVCodecContext *avctx, vpx_codec_caps_t codec_caps,
         *img_fmt = VPX_IMG_FMT_I440;
         return 0;
 #endif
+    case AV_PIX_FMT_GBRP:
     case AV_PIX_FMT_YUV444P:
         enccfg->g_profile = 1;
         *img_fmt = VPX_IMG_FMT_I444;

comment:2 by Hendrik, 8 years ago

On top of that patch, set_colorspace should probably automatically use RGB when a RGB pixfmt is used, not only when color_space is set.

comment:3 by Cigaes, 8 years ago

The patch works with an extra bit to add GBRP to the list of supported formats. Still, Hendrik is right, there should be automatic switch or at least a consistency check. And the high bit depths are probably missing.

comment:4 by Carl Eugen Hoyos, 8 years ago

Imo, the issue is that no encoder should have a private option colorspace.

Thanks to pkg-config=false I was able to test myself: Where did you add the pix_fmt? I didn't find the list yet...

in reply to:  4 ; comment:5 by Hendrik, 8 years ago

Replying to cehoyos:

Imo, the issue is that no encoder should have a private option colorspace.

It doesn't, it uses avctx->colorspace which -colorspace sets.

comment:6 by Cigaes, 8 years ago

As Hendrik pointed, it is not a private option, the documentation is slightly misleading.

The list is in vpx.c, I have posted the patch on the mailing-list.

in reply to:  5 comment:7 by Carl Eugen Hoyos, 8 years ago

Replying to heleppkes:

Replying to cehoyos:

Imo, the issue is that no encoder should have a private option colorspace.

It doesn't, it uses avctx->colorspace which -colorspace sets.

True, thanks.

comment:8 by Carl Eugen Hoyos, 8 years ago

Reproduced by developer: set

comment:9 by Carl Eugen Hoyos, 8 years ago

Resolution: fixed
Status: openclosed
Note: See TracTickets for help on using tickets.