Opened 3 years ago

Last modified 5 days ago

#6276 new defect

h264_vaapi & nvenc do not preserve aspect ratio changes

Reported by: Tom_B Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: vaapi nvenc aspect
Cc: zhuqingliang, linjie.fu@intel.com Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

ffmpeg output:

ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers

built with gcc 6.3.1 (GCC) 20170109
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-netcdf --enable-shared --enable-version3 --enable-x11grab
libavutil 55. 34.101 / 55. 34.101
libavcodec 57. 64.101 / 57. 64.101
libavformat 57. 56.101 / 57. 56.101
libavdevice 57. 1.100 / 57. 1.100
libavfilter 6. 65.100 / 6. 65.100
libavresample 3. 1. 0 / 3. 1. 0
libswscale 4. 2.100 / 4. 2.100
libswresample 2. 3.100 / 2. 3.100
libpostproc 54. 1.100 / 54. 1.100

Command:

ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi -i input.ts -y -g 30 -vf 'format=nv12,hwupload' -c:v h264_vaapi -qp 20 -sn out.ts

The input file is available at https://r.je/tmp/input.ts (apologies this is a 900mb file) with a change in aspect ratio from 16:9 to 4:3 at 3:28

When using h264_vaapi as the codec, it doens't maintain the aspect ratio change, however the aspect ratio change is maintained if using libx264

Using libx264, the aspect change is copied into the output file.

Note: The same bug happens when using h264_nvenc.

Attachments (1)

aspect_ratio_change_cut.ts (2.4 MB) - added by cehoyos 11 months ago.
Sample from ticket #3631

Change History (15)

comment:1 Changed 11 months ago by zhuqingliang

I try to reproduce this issue on my platform , but can't download https://r.je/tmp/input.ts , can you upload it again?

comment:2 follow-up: Changed 11 months ago by cehoyos

  • Component changed from ffmpeg to undetermined
  • Keywords ratio removed

Looks a little invalid as two bugs are reported, neither against current FFmpeg git head...

Changed 11 months ago by cehoyos

Sample from ticket #3631

comment:3 in reply to: ↑ 2 ; follow-up: Changed 11 months ago by zhuqingliang

Replying to cehoyos:

I test with

ffmpeg  -i aspect_ratio_change_cut.ts -vf fps=1 out_%3d.jpeg

and the out_0001.jpeg out_0002.jpeg out_0003.jpeg out_0004.jpeg is 720x576 , is it means this issue persists in master?

comment:4 in reply to: ↑ 3 ; follow-up: Changed 11 months ago by cehoyos

Replying to zhuqingliang:

Replying to cehoyos:

I test with

ffmpeg  -i aspect_ratio_change_cut.ts -vf fps=1 out_%3d.jpeg

The command line above looks different, what are you trying to test?

comment:5 in reply to: ↑ 4 Changed 11 months ago by zhuqingliang

I try to reproduce this issue: "h264_vaapi & nvenc do not preserve aspect ratio changes".

comment:6 follow-up: Changed 11 months ago by cehoyos

  • Cc zhuqingliang added
  • Component changed from undetermined to avcodec

Then why did you test with a different command line?

comment:7 in reply to: ↑ 6 Changed 9 months ago by zhuqingliang

Replying to cehoyos:

Then why did you test with a different command line?

because when I tried with

ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi -i aspect_ratio_change_cut.ts -y -g 30 -vf 'format=nv12,hwupload' -c:v h264_vaapi -qp 20 -sn out.ts 

failed with error message

[AVHWFramesContext @ 0x556ae298be00] Failed to sync surface 0x4: 23 (internal decoding error).
[mpeg2video @ 0x556ae2707440] Failed to transfer data to output frame: -5.

comment:8 Changed 12 days ago by yurtesen

  • Keywords amd added
  • Priority changed from normal to important
  • Version changed from git-master to 4.1.4

Even though similar problem exists with AMD Hardware. @cehoyos marked it as duplicate.

There is also information in the so called duplicate #8376 ticket and smaller test files and it is tested against 4.x master quite recently.

Last edited 10 days ago by yurtesen (previous) (diff)

comment:9 Changed 12 days ago by cehoyos

  • Priority changed from important to normal
  • Version changed from 4.1.4 to git-master

comment:10 Changed 12 days ago by cehoyos

  • Keywords amd removed

comment:11 Changed 10 days ago by fulinjie

  • Cc linjie.fu@intel.com added
  • Reproduced by developer set

The sync Surface error seems to be relevant with the input clips.
The last several frames of aspect_ratio_change_cut.ts is not good.

Simply cut these frames and the reported description could be observed.

ffmpeg -i aspect_ratio_change_cut.ts -c:v copy -vframes 80 80_aspect_ratio_change_cut.h264

No aspect ratio change with vaapi encode:

ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi  -i 80_aspect_ratio_change_cut.h264  -y -g 30 -vf 'format=nv12,hwupload' -c:v h264_vaapi -qp 20 -sn out.ts

Aspect ratio change observed with x264:

ffmpeg -hwaccel vaapi  -i 80_aspect_ratio_change_cut.h264  -y -g 30  -c:v libx264 -qp 20 -sn out_x264.ts

In libx264, reconfig_encoder() modified the SPS during encoding procedure when sample_aspect_ratio changes.

https://github.com/fulinjie/ffmpeg/blob/87ddf9f1ef17726fd4235f2e7aed8334d0ff231b/libavcodec/libx264.c#L198

Similar behavior could be added in VAAPI.

comment:12 follow-up: Changed 9 days ago by fulinjie

Hi yurtesen/Carl,

This patch helps to preserve the aspect ratio changes for VAAPI:
https://patchwork.ffmpeg.org/patch/16245/

Would you please help to verify?

BTW, similar issue existed in QSV. It could be addressed next.

ffmpeg -hwaccel vaapi -i 80_aspect_ratio_change_cut.h264 -y -g 30 -c:v h264_qsv -qp 20 -sn qsv.ts

comment:13 in reply to: ↑ 12 Changed 5 days ago by tvsht

Replying to fulinjie:

Hi yurtesen/Carl,

This patch helps to preserve the aspect ratio changes for VAAPI:
https://patchwork.ffmpeg.org/patch/16245/

Would you please help to verify?

BTW, similar issue existed in QSV. It could be addressed next.

ffmpeg -hwaccel vaapi -i 80_aspect_ratio_change_cut.h264 -y -g 30 -c:v h264_qsv -qp 20 -sn qsv.ts

Patched it and it still ends mangled [amdgpu]

sample file: [40MB] https://send.firefox.com/download/c48ebf7304f8a60f/#i5ZJ3Zdz1rjnz8lBxgjeaQ
end result: [27MB] https://send.firefox.com/download/42933b87d649abee/#2CJQHnxFiF6FV8F1PnHsIw

comment:14 Changed 5 days ago by tvsht

another one

sample: https://send.firefox.com/download/59da97f53074be4e/#wCPgaoWYuGNypLxrW7-NzA
output: https://send.firefox.com/download/456bfea4d1f3ef2e/#ev4es56EYiGStN7SnvQgUA

changes from x1036 to x1040 and introduces green line at the bottom
forcing size via vaapi_scale doesn't work and introduces additional glitch in the beginning

-hwaccel_output_format vaapi \
-c:v hevc_vaapi -qp 26 \
-c:a copy -c:s copy -f matroska

Posting here since it was marked as duplicate of this #8376

Note: See TracTickets for help on using tickets.