Opened 5 years ago

Last modified 3 years ago

#6276 new defect

h264_vaapi & nvenc do not preserve aspect ratio changes

Reported by: TomB Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: vaapi nvenc aspect
Cc: a, linjie.fu@intel.com Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

ffmpeg output:

ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers

built with gcc 6.3.1 (GCC) 20170109
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-netcdf --enable-shared --enable-version3 --enable-x11grab
libavutil 55. 34.101 / 55. 34.101
libavcodec 57. 64.101 / 57. 64.101
libavformat 57. 56.101 / 57. 56.101
libavdevice 57. 1.100 / 57. 1.100
libavfilter 6. 65.100 / 6. 65.100
libavresample 3. 1. 0 / 3. 1. 0
libswscale 4. 2.100 / 4. 2.100
libswresample 2. 3.100 / 2. 3.100
libpostproc 54. 1.100 / 54. 1.100

Command:

ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi -i input.ts -y -g 30 -vf 'format=nv12,hwupload' -c:v h264_vaapi -qp 20 -sn out.ts

The input file is available at https://r.je/tmp/input.ts (apologies this is a 900mb file) with a change in aspect ratio from 16:9 to 4:3 at 3:28

When using h264_vaapi as the codec, it doens't maintain the aspect ratio change, however the aspect ratio change is maintained if using libx264

Using libx264, the aspect change is copied into the output file.

Note: The same bug happens when using h264_nvenc.

Attachments (1)

aspect_ratio_change_cut.ts (2.4 MB ) - added by Carl Eugen Hoyos 4 years ago.
Sample from ticket #3631

Change History (15)

comment:1 by a, 4 years ago

I try to reproduce this issue on my platform , but can't download https://r.je/tmp/input.ts , can you upload it again?

comment:2 by Carl Eugen Hoyos, 4 years ago

Component: ffmpegundetermined
Keywords: ratio removed

Looks a little invalid as two bugs are reported, neither against current FFmpeg git head...

by Carl Eugen Hoyos, 4 years ago

Attachment: aspect_ratio_change_cut.ts added

Sample from ticket #3631

in reply to:  2 ; comment:3 by a, 4 years ago

Replying to cehoyos:

I test with

ffmpeg  -i aspect_ratio_change_cut.ts -vf fps=1 out_%3d.jpeg

and the out_0001.jpeg out_0002.jpeg out_0003.jpeg out_0004.jpeg is 720x576 , is it means this issue persists in master?

in reply to:  3 ; comment:4 by Carl Eugen Hoyos, 4 years ago

Replying to zhuqingliang:

Replying to cehoyos:

I test with

ffmpeg  -i aspect_ratio_change_cut.ts -vf fps=1 out_%3d.jpeg

The command line above looks different, what are you trying to test?

in reply to:  4 comment:5 by a, 4 years ago

I try to reproduce this issue: "h264_vaapi & nvenc do not preserve aspect ratio changes".

comment:6 by Carl Eugen Hoyos, 3 years ago

Cc: a added
Component: undeterminedavcodec

Then why did you test with a different command line?

in reply to:  6 comment:7 by a, 3 years ago

Replying to cehoyos:

Then why did you test with a different command line?

because when I tried with

ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi -i aspect_ratio_change_cut.ts -y -g 30 -vf 'format=nv12,hwupload' -c:v h264_vaapi -qp 20 -sn out.ts 

failed with error message

[AVHWFramesContext @ 0x556ae298be00] Failed to sync surface 0x4: 23 (internal decoding error).
[mpeg2video @ 0x556ae2707440] Failed to transfer data to output frame: -5.

comment:8 by Evren Yurtesen, 3 years ago

Keywords: amd added
Priority: normalimportant
Version: git-master4.1.4

Even though similar problem exists with AMD Hardware. @cehoyos marked it as duplicate.

There is also information in the so called duplicate #8376 ticket and smaller test files and it is tested against 4.x master quite recently.

Last edited 3 years ago by Evren Yurtesen (previous) (diff)

comment:9 by Carl Eugen Hoyos, 3 years ago

Priority: importantnormal
Version: 4.1.4git-master

comment:10 by Carl Eugen Hoyos, 3 years ago

Keywords: amd removed

comment:11 by Linjie.Fu, 3 years ago

Cc: linjie.fu@intel.com added
Reproduced by developer: set

The sync Surface error seems to be relevant with the input clips.
The last several frames of aspect_ratio_change_cut.ts is not good.

Simply cut these frames and the reported description could be observed.

ffmpeg -i aspect_ratio_change_cut.ts -c:v copy -vframes 80 80_aspect_ratio_change_cut.h264

No aspect ratio change with vaapi encode:

ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi  -i 80_aspect_ratio_change_cut.h264  -y -g 30 -vf 'format=nv12,hwupload' -c:v h264_vaapi -qp 20 -sn out.ts

Aspect ratio change observed with x264:

ffmpeg -hwaccel vaapi  -i 80_aspect_ratio_change_cut.h264  -y -g 30  -c:v libx264 -qp 20 -sn out_x264.ts

In libx264, reconfig_encoder() modified the SPS during encoding procedure when sample_aspect_ratio changes.

https://github.com/fulinjie/ffmpeg/blob/87ddf9f1ef17726fd4235f2e7aed8334d0ff231b/libavcodec/libx264.c#L198

Similar behavior could be added in VAAPI.

comment:12 by Linjie.Fu, 3 years ago

Hi yurtesen/Carl,

This patch helps to preserve the aspect ratio changes for VAAPI:
https://patchwork.ffmpeg.org/patch/16245/

Would you please help to verify?

BTW, similar issue existed in QSV. It could be addressed next.

ffmpeg -hwaccel vaapi -i 80_aspect_ratio_change_cut.h264 -y -g 30 -c:v h264_qsv -qp 20 -sn qsv.ts

in reply to:  12 comment:13 by tvsht, 3 years ago

Replying to fulinjie:

Hi yurtesen/Carl,

This patch helps to preserve the aspect ratio changes for VAAPI:
https://patchwork.ffmpeg.org/patch/16245/

Would you please help to verify?

BTW, similar issue existed in QSV. It could be addressed next.

ffmpeg -hwaccel vaapi -i 80_aspect_ratio_change_cut.h264 -y -g 30 -c:v h264_qsv -qp 20 -sn qsv.ts

Patched it and it still ends mangled [amdgpu]

sample file: [40MB] https://send.firefox.com/download/c48ebf7304f8a60f/#i5ZJ3Zdz1rjnz8lBxgjeaQ
end result: [27MB] https://send.firefox.com/download/42933b87d649abee/#2CJQHnxFiF6FV8F1PnHsIw

comment:14 by tvsht, 3 years ago

another one

sample: https://send.firefox.com/download/59da97f53074be4e/#wCPgaoWYuGNypLxrW7-NzA
output: https://send.firefox.com/download/456bfea4d1f3ef2e/#ev4es56EYiGStN7SnvQgUA

changes from x1036 to x1040 and introduces green line at the bottom
forcing size via vaapi_scale doesn't work and introduces additional glitch in the beginning

-hwaccel_output_format vaapi \
-c:v hevc_vaapi -qp 26 \
-c:a copy -c:s copy -f matroska

Posting here since it was marked as duplicate of this #8376

Note: See TracTickets for help on using tickets.