[regression] jpeg autorotate breaks vaapi 422 jpeg decode

[U. Artie Eoff]

Summary of the bug:

Vaapi jpeg decode plugin can't decode 422 jpeg image since ​http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=e93c9986027d17917c3b4f533b28ee4a2ce7cd4c

Format can't be converted between transpose and autoscale filters.

Only workaround is to disable auto rotate -autorotate 0.

How to reproduce:

$ ffmpeg -hwaccel vaapi -init_hw_device vaapi=hw:/dev/dri/renderD128 \
 -hwaccel_output_format vaapi -v verbose -i some-422.jpg \
 -vf 'hwdownload,format=yuv422p' -pix_fmt yuv422p \
 -f rawvideo -vframes 1 -y output.yuv

ffmpeg version N-103752-g59719a905c5e Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 9 (GCC)
  configuration: --prefix=/home/uaeoff/Work/workspace/media/install --disable-static --enable-shared --enable-libdrm --enable-vaapi --enable-libmfx --disable-amf --disable-audiotoolbox --disable-cuda --disable-cuda-sdk --disable-cuvid --disable-d3d11va --disable-dxva2 --disable-libnpp --disable-mmal --disable-nvdec --disable-nvenc --disable-omx --disable-omx-rpi --disable-rkmpp --disable-v4l2-m2m --disable-vdpau --disable-videotoolbox --enable-gpl --enable-libx264 --enable-libx265
  libavutil      57.  6.100 / 57.  6.100
  libavcodec     59.  9.100 / 59.  9.100
  libavformat    59.  5.100 / 59.  5.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8.  9.100 /  8.  9.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
[AVHWDeviceContext @ 0xc9c040] libva: VA-API version 1.13.0
[AVHWDeviceContext @ 0xc9c040] libva: User environment variable requested driver 'iHD'
[AVHWDeviceContext @ 0xc9c040] libva: Trying to open /home/uaeoff/Work/workspace/media/install/lib/dri/iHD_drv_video.so
[AVHWDeviceContext @ 0xc9c040] libva: Found init function __vaDriverInit_1_13
[AVHWDeviceContext @ 0xc9c040] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0xc9c040] Initialised VAAPI connection: version 1.13
[AVHWDeviceContext @ 0xc9c040] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 21.3.4 ().
[AVHWDeviceContext @ 0xc9c040] Driver not found in known nonstandard list, using standard behaviour.
Input #0, image2, from 'some-422.jpg':
  Duration: 00:00:00.04, start: 0.000000, bitrate: 217499 kb/s
  Stream #0:0: Video: mjpeg (Baseline), 1 reference frame, yuvj422p(pc, bt470bg/unknown/unknown, center), 2048x1536, 25 fps, 25 tbr, 25 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg (native) -> rawvideo (native))
Press [q] to stop, [?] for help
[graph 0 input from stream 0:0 @ 0xeb4b40] w:2048 h:1536 pixfmt:vaapi tb:1/25 fr:25/1 sar:0/1
[auto_scale_0 @ 0xeb8d40] w:iw h:ih flags:'' interl:0
[Parsed_hwdownload_0 @ 0xe51a40] auto-inserting filter 'auto_scale_0' between the filter 'transpose' and the filter 'Parsed_hwdownload_0'
Impossible to convert between the formats supported by the filter 'transpose' and the filter 'auto_scale_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:0
[AVIOContext @ 0xcec300] Statistics: 0 seeks, 0 writeouts
[AVIOContext @ 0xce4380] Statistics: 1087496 bytes read, 0 seeks
Conversion failed!

[U. Artie Eoff]

related #6945

[U. Artie Eoff]

hmm... 420 jpeg might be broken, too, with vaapi if file has exif orientation.

[James]

Replying to U. Artie Eoff:

hmm... 420 jpeg might be broken, too, with vaapi if file has exif orientation.

Yes, pretty much any jpeg sample with orientation exif and any h264 stream with Display Orientation SEI will be affected. What i did was make ffmpeg look at codec exported display matrix instead of only container exported.

Does the following fix it for you?

diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
index da0d4faf54..cf52cc03ba 100644
--- a/fftools/ffmpeg_filter.c
+++ b/fftools/ffmpeg_filter.c
@@ -699,6 +699,7 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter,
 {
     AVFilterContext *last_filter;
     const AVFilter *buffer_filt = avfilter_get_by_name("buffer");
+    const AVPixFmtDescriptor *desc;
     InputStream *ist = ifilter->ist;
     InputFile     *f = input_files[ist->file_index];
     AVRational tb = ist->framerate.num ? av_inv_q(ist->framerate) :
@@ -756,7 +757,8 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter,
     av_freep(&par);
     last_filter = ifilter->filter;
 
-    if (ist->autorotate) {
+    desc = av_pix_fmt_desc_get(ifilter->format);
+    if (ist->autorotate && desc && !(desc->flags & AV_PIX_FMT_FLAG_HWACCEL)) {
         int32_t *displaymatrix = ifilter->displaymatrix;
         double theta;
 

[U. Artie Eoff]

Replying to James:

Replying to U. Artie Eoff:

hmm... 420 jpeg might be broken, too, with vaapi if file has exif orientation.

Yes, pretty much any jpeg sample with orientation exif and any h264 stream with Display Orientation SEI will be affected. What i did was make ffmpeg look at codec exported display matrix instead of only container exported.

Does the following fix it for you?

diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
index da0d4faf54..cf52cc03ba 100644
--- a/fftools/ffmpeg_filter.c
+++ b/fftools/ffmpeg_filter.c
@@ -699,6 +699,7 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter,
 {
     AVFilterContext *last_filter;
     const AVFilter *buffer_filt = avfilter_get_by_name("buffer");
+    const AVPixFmtDescriptor *desc;
     InputStream *ist = ifilter->ist;
     InputFile     *f = input_files[ist->file_index];
     AVRational tb = ist->framerate.num ? av_inv_q(ist->framerate) :
@@ -756,7 +757,8 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter,
     av_freep(&par);
     last_filter = ifilter->filter;
 
-    if (ist->autorotate) {
+    desc = av_pix_fmt_desc_get(ifilter->format);
+    if (ist->autorotate && desc && !(desc->flags & AV_PIX_FMT_FLAG_HWACCEL)) {
         int32_t *displaymatrix = ifilter->displaymatrix;
         double theta;
 

Yes, this patch fixes it (i.e. by avoiding hwaccel formats).

[James]

Fixed in ed1a70469e526bd3a4122242b66d73914743094f