Opened 8 years ago

Last modified 2 months ago

#6260 reopened defect

NVENC H.264 encoding causes aliasing / stair-stepping colors, worse on interlaced than progressive

Reported by: alexpigment Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: NVENC
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: yes

Description

Summary of the bug:

When encoding to H.264 with the NVENC encoder, I'm seeing a pronounced aliasing or stair-stepping effect that happens on the colors (chroma-only?) during motion. This occurs on progressive encodes, but is worse on interlaced encodes. I have verified this behavior on both an Nvidia 1060 and an Nvidia 750 while using the newest static Win64 build from Zeranoe as well as a build from January.

To rule out a problem with the GPU, driver, or NVENC itself, I used Staxrip x264 1.4.0.0 to encode with NVENC H.264 on 2 samples and saw no issues. Therefore, this issue appears to be specific to the FFMPEG implementation of NVENC. FFMPEG x264 does not suffer from this issue either.

How to reproduce:

% ffmpeg -i interlacedsample.mp4 -c:v h264_nvenc -b:v 20000000 -flags +ilme+ildct interlacedoutput.mp4

View the file in a player that supports deinterlacing (I'm using WMP 12 for consistency). Notice that the colors of the rings have massive stair-stepping around the curves, particularly in the lower right where the red and green intersect.

As mentioned above, this also occurs on progressive, albeit to a lesser but still very problematic degree.

% ffmpeg -i progressivesample.mp4 -c:v h264_nvenc -b:v 20000000 progressiveoutput.mp4

The same problem occurs here, most notably at hte top of the blue right and at the intersection of the green and red rings at the bottom left.

ffmpeg version N-84348-gdb7a05d
built on 2017-03-21

Attachments (5)

interlacedsample.mp4 (2.1 MB ) - added by alexpigment 8 years ago.
This is the input file as used in the first part of the reproduction steps.
progressivesample.mp4 (1.5 MB ) - added by alexpigment 8 years ago.
This is the input file as used in the second part of the reproduction steps.
interlacedoutput.mp4 (2.4 MB ) - added by alexpigment 8 years ago.
This is the problematic output file that results from the first part of the reproduction steps (bitrate lowered for file size considerations).
progressiveoutput.mp4 (2.1 MB ) - added by alexpigment 8 years ago.
This is the problematic output file that results from the second part of the reproduction steps (bitrate lowered for file size considerations).
stairstepping.png (1.9 MB ) - added by alexpigment 8 years ago.
Screenshot of "interlacedoutput.mp4" created in build after 'fix' was committed; still seeing the same issue. See bottom right corner where green and red rings meet. Same is true for yellow and blue rings.

Change History (13)

by alexpigment, 8 years ago

Attachment: interlacedsample.mp4 added

This is the input file as used in the first part of the reproduction steps.

by alexpigment, 8 years ago

Attachment: progressivesample.mp4 added

This is the input file as used in the second part of the reproduction steps.

by alexpigment, 8 years ago

Attachment: interlacedoutput.mp4 added

This is the problematic output file that results from the first part of the reproduction steps (bitrate lowered for file size considerations).

by alexpigment, 8 years ago

Attachment: progressiveoutput.mp4 added

This is the problematic output file that results from the second part of the reproduction steps (bitrate lowered for file size considerations).

comment:1 by Carl Eugen Hoyos, 8 years ago

Component: ffmpegundetermined
Keywords: aliasing interlaced progressive chroma removed
Priority: importantnormal

comment:2 by dpavlekovic, 8 years ago

Hi, I'm experiencing the same problem, I originally thought it's the nvenc issue so thanks for reporting. Today I found that this is related to padding of input surface width/height to be divisible by 32 in libavcodec/nvenc.c:nvenc_alloc_surface:

        allocSurf.width = (avctx->width + 31) & ~31;
        allocSurf.height = (avctx->height + 31) & ~31;

If I remove this adjustment then the output looks OK (1920x1080 source, would be padded to 1920x1088). I've no experience though with NVEnc APIs, so not sure if this is a right thing to do for all cases.

Alternatively, keeping the padding and setting the input picture width/height to padded input surface dimensions rather than original frame dims in ff_nvenc_encode_frame seems to work as well:

@@ -1737,8 +1790,8 @@ int ff_nvenc_encode_frame(AVCodecContext *avctx, AVPacket *pkt,

         pic_params.inputBuffer = inSurf->input_surface;
         pic_params.bufferFmt = inSurf->format;
-        pic_params.inputWidth = avctx->width;
-        pic_params.inputHeight = avctx->height;
+        pic_params.inputWidth = inSurf->width;
+        pic_params.inputHeight = inSurf->height;
         pic_params.inputPitch = inSurf->pitch;
         pic_params.outputBitstream = inSurf->output_surface;
Last edited 8 years ago by dpavlekovic (previous) (diff)

comment:3 by Timo R., 8 years ago

Analyzed by developer: set
Component: undeterminedavcodec
Reproduced by developer: set
Resolution: fixed
Status: newclosed
Version: unspecifiedgit-master

Fixed in a1652aca7e892ea4899df894feaa87ba4170e06c

Also backported to all releases since 3.1.

Last edited 8 years ago by Timo R. (previous) (diff)

comment:4 by alexpigment, 8 years ago

This doesn't appear to be addressed on my end. I've tested the latest Zeranoe build, but I also compiled a nightly Win32 build with the relevant patch applied. See "stairstepping.png" attachment.

Did either dpavlekovic or oromit test the reproduction steps to make sure the fix worked in that scenario?

by alexpigment, 8 years ago

Attachment: stairstepping.png added

Screenshot of "interlacedoutput.mp4" created in build after 'fix' was committed; still seeing the same issue. See bottom right corner where green and red rings meet. Same is true for yellow and blue rings.

comment:5 by alexpigment, 8 years ago

Resolution: fixed
Status: closedreopened

comment:6 by tbucher, 8 years ago

I ran into the same issue with a build from a couple of weeks ago, haven't updated since to verify whether the mentioned fix has changed anything.

At that time I found out that the aliasing artifacts manifest themselves while using nvenc with yuv420p colorspace, enforcing it to nv12 fixed it.

Can you confirm that nv12 works properly for you as well? If yes, it might be a lead for developers to follow.

comment:7 by alexpigment, 8 years ago

Yes, adding -pix_fmt nv12 does indeed fix the problem! Thanks for that tip, tbucher.

This does make me wonder what to do with this issue. Do we consider this an issue with yuv420p that needs to be addressed, or should nv12 become the default selection when nothing is explicitly set? I know that with h264_qsv, yuv420p is not even an option and it will fall back to nv12 even if you specify yuv420p explictly.

comment:8 by Balling, 2 months ago

Well, whatever the issue was there it no longer happens or -pix_fmt nv12 does not change the produced file.

My card is Turing and so does not support interlaced (well, encoding).

Last edited 2 months ago by Balling (previous) (diff)
Note: See TracTickets for help on using tickets.