Opened 2 years ago

Closed 20 months ago

#9645 closed defect (wontfix)

Can't enable hardware decoders for H.264/AVC and H.265/HEVC without enabling software decoders

Reported by: Neal Gompa Owned by:
Priority: normal Component: avcodec
Version: unspecified Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:
In Fedora, we're trying to ship a build of FFmpeg that's maximally useful without shipping encumbered stuff. To that end, we want to be able to rely on built-in hardware codecs available on the user's system for codecs we can't ship (such as H264 (AVC) and HEVC (H.265)). Unfortunately, it seems that it doesn't work for all decoders but the V4L2-M2M and QSV ones. That means people who have NVDEC, VDPAU, and VAAPI offload for decoding cannot use them through ffmpeg.

A solution for this would require source level separation so that just enough stuff is available to leverage the hardware decoders, since we have to strip ffmpeg sources of stuff we can't ship before we upload it for build. My understanding is that the AV1 decoder has this separation (though we have AV1 fully turned on in FFmpeg in Fedora), so it's not entirely without precedent.

How to reproduce:

  1. Download the ffmpeg-5.0 release tarball
  2. Build ffmpeg with the following configure line:
    % ./configure --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 '--optflags=-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' '--extra-cflags=-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --disable-htmlpages --enable-pic --disable-stripping --enable-shared --disable-static --enable-gpl --enable-version3 --enable-libsmbclient --disable-openssl --enable-gnutls --enable-ladspa --enable-vulkan --disable-cuda-sdk --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcdio --enable-libdav1d --enable-libdc1394 --enable-libdrm --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librav1e --enable-librubberband --enable-libsvtav1 --enable-libsoxr --enable-libspeex --enable-libssh --enable-libsrt --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libv4l2 --enable-libvpx --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libzvbi --enable-lto --enable-libmfx --enable-vaapi --enable-vdpau --enable-muxers --enable-demuxers --disable-encoders --disable-decoders --disable-decoder=mpeg4,h263,h264,hevc,vc1 --enable-encoder=,libfdk_aac,ac3,apng,ass,ayuv,bmp,ffv1,ffvhuff,flac,gif,h263_v4l2m2m,h264_amf,h264_nvenc,h264_qsv,h264_v4l2m2m,h264_vaapi,hevc_amf,hevc_nvenc,hevc_qsv,hevc_v4l2m2m,hevc_vaapi,huffyuv,jpegls,jpeg2000,libaom,libaom_av1,libcodec2,libgsm,libmp3lame,libopenjpeg,libopus,librav1e,libschroedinger,libspeex,libsvtav1,libtheora,libtwolame,libvorbis,libvpx_vp8,libvpx_vp9,libwebp,libwebp_anim,mjpeg,mjpeg_qsv,mp2,mp2fixed,mpeg1video,mpeg2video,mpeg2_qsv,mpeg4_v4l2m2m,opus,pam,pbm,pcm_alaw,pcm_f32be,pcm_f32le,pcm_f64be,pcm_f64le,pcm_mulaw,pcm_s16be,pcm_s16be_planar,pcm_s16le,pcm_s16le_planar,pcm_s24be,pcm_s24le,pcm_s24le_planar,pcm_s32be,pcm_s32le,pcm_s32le_planar,pcm_s8,pcm_s8_planar,pcm_u16be,pcm_u16le,pcm_u24be,pcm_u24le,pcm_u32be,pcm_u32le,pcm_u8,pcx,pgm,pgmyuv,png,ppm,rawvideo,sgi,srt,ssa,sunrast,targa,text,tiff,v210,v308,v408,v410,vc1_qsv,vc1_v4l2m2m,vorbis,vp8_qsv,vp8_v4l2m2m,vp9_qsv,webvtt,wrapped_avframe,xbm,xwd,y41p,yuv4,zlib, --enable-decoder=,libfdk_aac,ac3,ansi,apng,ass,av1_qsv,ayuv,bmp,dirac,exr,ffv1,ffvhuff,ffwavesynth,flac,gif,gsm,h263_v4l2m2m,h264_nvdec,h264_vdpau,h264_vaapi,h264_v4l2m2m,h264_qsv,hevc_nvdec,hevc_vdpau,hevc_vaapi,hevc_v4l2m2m,hevc_qsv,huffyuv,ilbc,jpeg2000,libaom,libaom_av1,libcelt,libcodec2,libdav1d,libgsm,libopenjpeg,libopus,libschroedinger,libspeex,libvorbis,libvpx_vp8,libvpx_vp9,mjpeg,mjpeg_qsv,mp1,mp1float,mp2,mp2float,mp3,mp3float,mpeg1video,mpeg1_v4l2m2m,mpeg2video,mpeg2_qsv,mpeg2_v4l2m2m,mpeg4_v4l2m2m,opus,pam,pbm,pcm_alaw,pcm_bluray,pcm_dvd,pcm_f32be,pcm_f32le,pcm_f64be,pcm_f64le,pcm_mulaw,pcm_s16be,pcm_s16be_planar,pcm_s16le,pcm_s16le_planar,pcm_s24be,pcm_s24le,pcm_s24le_planar,pcm_s32be,pcm_s32le,pcm_s32le_planar,pcm_s8,pcm_s8_planar,pcm_u16be,pcm_u16le,pcm_u24be,pcm_u24le,pcm_u32be,pcm_u32le,pcm_u8,pcx,pgm,pgmyuv,pgssub,pgx,png,ppm,rawvideo,sgi,srt,ssa,sunrast,targa,text,theora,tiff,v210,v210x,v308,v408,v410,vc1_qsv,vc1_v4l2m2m,vorbis,vp3,vp5,vp6,vp6a,vp6f,vp8,vp8_qsv,vp8_v4l2m2m,vp9,vp9_qsv,vp9_v4l2m2m,webp,webvtt,wrapped_avframe,xbm,xwd,y41p,yuv4,zlib,
    
  3. Note that the resulting ffmpeg lacks a number of hardware decoders enabled at configure time:
    % ffmpeg -decoders | grep h264
    [...]
     V..... h264_v4l2m2m         V4L2 mem2mem H.264 decoder wrapper (codec h264)
     V....D h264_qsv             H264 video (Intel Quick Sync Video acceleration) (codec h264)
    % ffmpeg -decoders | grep hevc
    [...]
     V....D hevc_qsv             HEVC video (Intel Quick Sync Video acceleration) (codec hevc)
     V..... hevc_v4l2m2m         V4L2 mem2mem HEVC decoder wrapper (codec hevc)
    

Change History (6)

comment:1 by Balling, 2 years ago

codecs we can't ship (such as H264 (AVC) and

Of course you can ship h264, you cannot ship libx264. NVDEC is not going to work without h264, just like the encoder. NEVER. And maybe cuvid decoders/encoders may allow this, but I doubt it and they are deprecated anyway.

in reply to:  1 comment:2 by Neal Gompa, 2 years ago

Replying to Balling:

codecs we can't ship (such as H264 (AVC) and

Of course you can ship h264, you cannot ship libx264. NVDEC is not going to work without h264, just like the encoder. NEVER. And maybe cuvid decoders/encoders may allow this, but I doubt it and they are deprecated anyway.

The hardware encoders work just fine with having the ffmpeg h264 and hevc software stuff disabled and stripped.

% ffmpeg -encoders | grep h264
[...]
 V....D h264_amf             AMD AMF H.264 Encoder (codec h264)
 V....D h264_nvenc           NVIDIA NVENC H.264 encoder (codec h264)
 V..... h264_qsv             H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264)
 V..... h264_v4l2m2m         V4L2 mem2mem H.264 encoder wrapper (codec h264)
 V....D h264_vaapi           H.264/AVC (VAAPI) (codec h264)
% ffmpeg -encoders | grep hevc
[...]
 V....D hevc_amf             AMD AMF HEVC encoder (codec hevc)
 V....D hevc_nvenc           NVIDIA NVENC hevc encoder (codec hevc)
 V..... hevc_qsv             HEVC (Intel Quick Sync Video acceleration) (codec hevc)
 V..... hevc_v4l2m2m         V4L2 mem2mem HEVC encoder wrapper (codec hevc)
 V....D hevc_vaapi           H.265/HEVC (VAAPI) (codec hevc)

comment:3 by mkver, 2 years ago

AV1 is not a real precedent for this; we just don't have a native AV1 decoder yet.

comment:4 by Balling, 2 years ago

Okay, but the nvdec hwaccel is using h264 to parse everything... And as for nvenc you are being very funny right now.

https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/nvenc_h264.c

in reply to:  3 comment:5 by Balling, 2 years ago

Replying to mkver:

AV1 is not a real precedent for this; we just don't have a native AV1 decoder yet.

We do have an OBU parser though. And h.274 film grain and specific to av1 too (?). And what not.

Last edited 2 years ago by Balling (previous) (diff)

comment:6 by Carl Eugen Hoyos, 20 months ago

Resolution: wontfix
Status: newclosed

If the described behaviour changes (which is not so unlikely), it will not be related to the reasoning of this ticket.

Note: See TracTickets for help on using tickets.