#10221 closed defect (fixed)

Null pointer when nvenc gets aborted

Reported by: Daniel Stankewitz Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords:
Cc: Daniel Stankewitz Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description (last modified by Daniel Stankewitz)

Summary of the bug:

libavcodec crashes if nvenc gets aborted with i.e. "No capable devices found" or any other error.

How to reproduce:

With any NVIDIA GPU that is not AV1 capable (because that'll create the error "No capable devices found") with this example.

% ffmpeg.exe -v trace -f lavfi -i smptebars=size=1920x1080 -t 5 -c:v av1_nvenc -b:v 500k -c:a none -pix_fmt yuv420p test.mp4


const AVCodec* codec = avcodec_find_encoder_by_name("av1_nvenc");
AVCodecContext* codecContext = avcodec_alloc_context3(codec);
codecContext->width = 320;
codecContext->height = 240;
codecContext->time_base = { 1, 25 };
codecContext->pix_fmt = AV_PIX_FMT_YUV420P;
int res = avcodec_open2(codecContext, codec, NULL);

ffmpeg version: a02e45a1f3cc6f07c7437c1e225dea2683f843cf

In ff_nvenc_encode_init the initialization order is:

if ((ret = nvenc_load_libraries(avctx)) < 0)
    return ret;

if ((ret = nvenc_setup_device(avctx)) < 0)
    return ret;

if ((ret = nvenc_setup_encoder(avctx)) < 0)
    return ret;

if ((ret = nvenc_setup_surfaces(avctx)) < 0)
    return ret;

... but in ff_nvenc_encode_close() it tries to work with fifo buffers that are still null because they'll get initialized in nvenc_setup_surfaces() later (after nvenc_setup_device()).

Change History (3)

comment:1 by Daniel Stankewitz, 14 months ago

Description: modified (diff)

comment:3 by Balling, 14 months ago

Resolution: fixed
Status: openclosed
Note: See TracTickets for help on using tickets.