Small MAX_SLICES makes d3d11va (AVC) not bitperfect and causes artifacts (but not CUDA/NVDEC)

[Balling]

Some time ago I saw this commit in Nev's fork of ffmpeg: ​https://gitea.1f0.de/LAV/FFmpeg/commit/a1246dd8ba2e5f56e83bf94bef86d1fca22adaed (patch fixes this bug)

Then I saw this: ​https://github.com/wang-bin/QtAV/issues/923 (that file does not warn that "MAX_SLICES should be increased", may be another bug)
And then I saw #628 and tested it. This is what I found:

-hwaccel cuda and -c:v h264_cuvid input option to NV12 are bitperfect with each other and bitperfect with Nev's fork d3d11va, but not bitperfect with HEAD of our master branch d3d11va and on a sample from ​https://github.com/wang-bin/QtAV/files/1132086/example.zip causes the artifacts mentioned there and on sample from #628 even worse artifacts are present. I will also point out that there is no difference with SW decoding of sample in #628 between the master and the fork which means the increase in SAMPLES in avc bitsream does not matter on that sample. Of course someone should still check the reference decoder from ​https://vcgit.hhi.fraunhofer.de/jvet
How to reproduce:

ffmpeg.exe -hwaccel nvdec -i error_4942.264 -f md5 -
ffmpeg version N-106825-g30a49a5230-20220507 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 11.2.0 (crosstool-NG 1.24.0.533_681aaef)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --enable-shared --disable-static --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20220507
  libavutil      57. 24.101 / 57. 24.101
  libavcodec     59. 27.100 / 59. 27.100
  libavformat    59. 23.100 / 59. 23.100
  libavdevice    59.  6.100 / 59.  6.100
  libavfilter     8. 38.100 /  8. 38.100
  libswscale      6.  6.100 /  6.  6.100
  libswresample   4.  6.100 /  4.  6.100
  libpostproc    56.  5.100 / 56.  5.100
Input #0, h264, from 'error_4942.264':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 352x288, 30 fps, 30 tbr, 1200k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, md5, to 'pipe:':
  Metadata:
    encoder         : Lavf59.23.100
  Stream #0:0: Video: rawvideo (NV12 / 0x3231564E), nv12(progressive), 352x288, q=2-31, 36495 kb/s, 30 fps, 30 tbn
    Metadata:
      encoder         : Lavc59.27.100 rawvideo
[h264 @ 000001115560d9c0] Possibly too many slices (110 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (111 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (112 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (116 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (117 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (118 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (119 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (120 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (121 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (122 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (123 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (124 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (99 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (100 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (32 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (33 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (34 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (35 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (36 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (37 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (38 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (39 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (40 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (41 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (42 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (43 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (44 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (45 >= 32), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 000001115560d9c0] Possibly too many slices (46 >= 32), increase MAX_SLICES and recompile if there are artifacts
...

MD5=ab07a0a0a82a2e9a6486a8b6994eb505
frame=  299 fps=0.0 q=-0.0 Lsize=       0kB time=00:00:09.96 bitrate=   0.0kbits/s speed=32.7x
video:44402kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

ffmpeg.exe -c:v h264_cuvid -i error_4942.264 -f md5 -
ffmpeg version N-106825-g30a49a5230-20220507 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 11.2.0 (crosstool-NG 1.24.0.533_681aaef)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --enable-shared --disable-static --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20220507
  libavutil      57. 24.101 / 57. 24.101
  libavcodec     59. 27.100 / 59. 27.100
  libavformat    59. 23.100 / 59. 23.100
  libavdevice    59.  6.100 / 59.  6.100
  libavfilter     8. 38.100 /  8. 38.100
  libswscale      6.  6.100 /  6.  6.100
  libswresample   4.  6.100 /  4.  6.100
  libpostproc    56.  5.100 / 56.  5.100
Input #0, h264, from 'error_4942.264':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 352x288, 30 fps, 30 tbr, 1200k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_cuvid) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, md5, to 'pipe:':
  Metadata:
    encoder         : Lavf59.23.100
  Stream #0:0: Video: rawvideo (NV12 / 0x3231564E), nv12(tv, progressive), 352x288 [SAR 1:1 DAR 11:9], q=2-31, 36495 kb/s, 30 fps, 30 tbn
    Metadata:
      encoder         : Lavc59.27.100 rawvideo
MD5=ab07a0a0a82a2e9a6486a8b6994eb505  0kB time=00:00:00.00 bitrate=N/A speed=   0x
frame=  299 fps=0.0 q=-0.0 Lsize=       0kB time=00:00:09.96 bitrate=   0.0kbits/s speed=43.7x
video:44402kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

ffmpeg.exe -hwaccel d3d11va -i error_4942.264 -f md5 -
...
MD5=2503d2e721d32182a09538f73d534255

And finaly Nev's fork:

Lav\ffmpeg.exe -hwaccel d3d11va -i error_4942.264 -f md5 -

ffmpeg version git-2022-01-04-176dfd865e-20220331 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 11.2.0 (crosstool-NG 1.24.0.533_681aaef)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --enable-shared --disable-static --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20220331
  libavutil      57. 16.100 / 57. 16.100
  libavcodec     59. 17.100 / 59. 17.100
  libavformat    59. 15.100 / 59. 15.100
  libavdevice    59.  3.100 / 59.  3.100
  libavfilter     8. 23.100 /  8. 23.100
  libswscale      6.  3.100 /  6.  3.100
  libswresample   4.  2.100 /  4.  2.100
  libpostproc    56.  2.100 / 56.  2.100
Input #0, h264, from 'error_4942.264':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 352x288, 30 fps, 30 tbr, 1200k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, md5, to 'pipe:':
  Metadata:
    encoder         : Lavf59.15.100
  Stream #0:0: Video: rawvideo (NV12 / 0x3231564E), nv12(progressive), 352x288, q=2-31, 36495 kb/s, 30 fps, 30 tbn
    Metadata:
      encoder         : Lavc59.17.100 rawvideo
MD5=ab07a0a0a82a2e9a6486a8b6994eb505  0kB time=00:00:00.00 bitrate=N/A speed=   0x
frame=  299 fps=0.0 q=-0.0 Lsize=       0kB time=00:00:09.96 bitrate=   0.0kbits/s speed=20.8x
video:44402kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

Same for the other sample. Same happens in VLC when you remux either sample to mp4 (also see mpv.com -hwdec=d3d11va error_4942.264 or mpv.com -hwdec=dxva2 error_4942.264 but not mpv.com -hwdec=nvdec -gpu-api=vulkan error_4942.264). Does not happen in Davinci since that uses directly nvdec which is not affected (I know since it also supports HW accel of HEVC 4:4:4 10 bit). dxva2 is also affected though on Nev's fork it is broken.
Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker. does not apply, patch is simple, but the bug may be somehow fixed differently, I am sure, since SW decoder MAX_SLICES do not effect anything.

[Balling]

Bug in VLC too:

​https://code.videolan.org/videolan/vlc/-/issues/26918

They also confirmed bug is fixed with this patch to 256.

[Balling]

that file does not warn that "MAX_SLICES should be increased", may be another bug

It is a bug. Had to compile reference decoder while changing the MAX_NUM_SLICES there (not ffmpeg's MAX_SLICES). Apparently that video has 41 slices (reference decoder does not require it to be a power of two). See ​https://code.videolan.org/videolan/vlc/-/issues/26918#note_324215

[Florian Grill]

Probably related to my recent bug report

​https://trac.ffmpeg.org/ticket/10374

[Balling]

Hey, Timo, any idea whether this cam be fixed before Christmas? Patch is there, you know.

[Balling]

Re: IRC

This is not a bug in software decoder, but A BUG that breaks perfectly capable of it HW decoders.

You do not have to do what Nev does: you can just patch dxva2_h264.c

dxva2_h264.c, so why is dxva2_h264 just not overriding this limit like the dxva2_hevc decoder for example?

[Timo R.]

I'd be more inclined to just get rid of the limit entirely, given it's not based on the reality that the codec has no limit.
But that's obviously a bigger undertaking.

For just dxva2_h264 it seems fairly simply though, just convert the two fixed size arrays into one dynamically growing one. Maybe with some pretty high upper limit, to avoid runaway memory allocations.

[Balling]

There is a limit in dxva2 decoder in nvidia driver, it cannot do ​https://code.videolan.org/-/project/435/uploads/e52babf681e96f453a151d98a65942c8/coow1212l.mp4

sample, but d3d11 can do it.

There is no strict limit in sw decoder, MAX_SLICES 32 means nothing, as there are 41 slices in ​https://github.com/wang-bin/QtAV/files/1132086/example.zip and yet it works just fine with software decoder and does not warn about anything.

In fact IT SAYS: increase MAX_SLICES and recompile if there are artifacts

[Timo R.]

The software decoder also has a fixed size array based on MAX_SLICES.
And then just does "& (MAX_SLICES -1)" when indexing into them.
So how badly it breaks depends on how all the slices interact.