Opened 2 months ago

Closed 2 months ago

#6592 closed defect (fixed)

DNxHD error during encoding: too many threads

Reported by: ralambda Owned by:
Priority: normal Component: undetermined
Version: git-master Keywords: DNxHD, too many threads
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary:
When encoding DNxHD mxf file the encoder crashes with the error "too many threads". I've reproduced this error with various input files of various resolutions and codecs including h264 (yuv420, high, 5.1), AVC Intra, and MPEG-2.

Error:

[dnxhd @ 0x3e92c00] too many threads
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

How to reproduce:

/usr/local/bin/ffmpeg -v 9 -loglevel 99 -y -r 23.976 -i input_file -c:v dnxhd -b:v 36M -pix_fmt:v yuv422p -s:v 1920x1080 -an output_file.mxf

ffmpeg version

ffmpeg version N-86980-g62b7553 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-11)
configuration: --prefix=/home/alchemist/ffmpeg_build --extra-cflags=-I/home/alchemist/ffmpeg_build/include --extra-ldflags='-L/home/alchemist/ffmpeg_build/lib -ldl' --bindir=/home/alchemist/bin --pkg-config-flags=--static --enable-gpl --enable-libfdk_aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree
libavutil      55. 73.100 / 55. 73.100
libavcodec     57.102.100 / 57.102.100
libavformat    57. 76.100 / 57. 76.100
libavdevice    57.  7.100 / 57.  7.100
libavfilter     6. 98.100 /  6. 98.100
libswscale      4.  7.102 /  4.  7.102
libswresample   2.  8.100 /  2.  8.100
libpostproc    54.  6.100 / 54.  6.100

Built on: CentOS 7 from source (Git master)

The command above worked successfully on the last version I had built. Unfortunately I don't have the build info for that version. I compiled from the git today (August 16, 2017) and now getting the error.

The issue might be related to this update: https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/9835ee60da31184dd6f6700328e75c78fa485efe

The terminal output is over 26,000 lines when using -v 9 -loglevel 99.
Here's the output once encoding starts:

[dnxhd @ 0x5ac2e80] cid 1253
[dnxhd @ 0x6018820] cid 1253
[dnxhd @ 0x61489c0] cid 1253
[dnxhd @ 0x6278d60] cid 1253
[dnxhd @ 0x63a8f00] cid 1253
[dnxhd @ 0x64d9140] cid 1253
[dnxhd @ 0x66093c0] cid 1253
[dnxhd @ 0x6739760] cid 1253
[dnxhd @ 0x6869900] cid 1253
[dnxhd @ 0x6999ca0] cid 1253
[dnxhd @ 0x6ac9e40] cid 1253
[dnxhd @ 0x6bfa1e0] cid 1253
[dnxhd @ 0x6d2a380] cid 1253
[dnxhd @ 0x6e5a720] cid 1253
[dnxhd @ 0x6f8a8c0] cid 1253
[dnxhd @ 0x70bac60] cid 1253
[dnxhd @ 0x71eae00] cid 1253
[dnxhd @ 0x731b1a0] cid 1253
[dnxhd @ 0x744b340] cid 1253
[dnxhd @ 0x757b6e0] cid 1253
[dnxhd @ 0x76ab880] cid 1253
[dnxhd @ 0x77dbc20] cid 1253
[dnxhd @ 0x790bdc0] cid 1253
[dnxhd @ 0x7a3c160] cid 1253
[dnxhd @ 0x7b6c300] cid 1253
[dnxhd @ 0x7c9c6a0] cid 1253
[dnxhd @ 0x7dcc840] cid 1253
[dnxhd @ 0x7efcbe0] cid 1253
[dnxhd @ 0x802cd80] cid 1253
[dnxhd @ 0x815d120] cid 1253
[dnxhd @ 0x828d2c0] cid 1253
[dnxhd @ 0x83bd660] cid 1253
[dnxhd @ 0x84ed800] cid 1253
[dnxhd @ 0x861dba0] cid 1253
[dnxhd @ 0x874dd40] cid 1253
[dnxhd @ 0x887e0e0] cid 1253
[dnxhd @ 0x89ae280] cid 1253
[dnxhd @ 0x8ade620] cid 1253
[dnxhd @ 0x8c0e7c0] cid 1253
[dnxhd @ 0x8d3eb60] cid 1253
[dnxhd @ 0x3e92c00] cid 1253
[dnxhd @ 0x3e92c00] too many threads
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[AVIOContext @ 0x3e96d80] Statistics: 0 seeks, 0 writeouts
[AVIOContext @ 0x3e710c0] Statistics: 2790563 bytes read, 1 seeks
Conversion failed!

Change History (6)

comment:1 Changed 2 months ago by ralambda

This was resolved by compiling version 3.3.3 Hilbert

built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-11)
configuration: --prefix=/home/user/ffmpeg_build --extra-cflags=-I/home/user/ffmpeg_build/include --extra-ldflags='-L/home/user/ffmpeg_build/lib -ldl' --bindir=/home/user/bin --pkg-config-flags=--static --enable-gpl --enable-nonfree --enable-libfdk_aac --enable-libfreetype --enable-libx264 --enable-libx265
libavutil      55. 58.100 / 55. 58.100
libavcodec     57. 89.100 / 57. 89.100
libavformat    57. 71.100 / 57. 71.100
libavdevice    57.  6.100 / 57.  6.100
libavfilter     6. 82.100 /  6. 82.100
libswscale      4.  6.100 /  4.  6.100
libswresample   2.  7.100 /  2.  7.100
libpostproc    54.  5.100 / 54.  5.100
Last edited 2 months ago by ralambda (previous) (diff)

comment:2 Changed 2 months ago by richardpl

Doesn't make sense, how many cores/CPUS do you have?
Also does this happens with DNxHD only?

Last edited 2 months ago by richardpl (previous) (diff)

comment:3 Changed 2 months ago by ralambda

The system has 20 physical cores (40 logical). It seems that each core is printed in the encoding output above.

This only happens with DNxHD encoding.

Important note: the DNxHD encoder is working on 6 and 8 core systems.

comment:4 Changed 2 months ago by richardpl

What happens if you explicitly set threads with '-threads' to some number less than 64?

comment:5 Changed 2 months ago by ralambda

It works if -threads less than or equal to 32 on the 20 core system It fails if -threads is greater than 32.

It's the same error when it fails when the -threads option is added: [dnxhd @ 0x319ec80] too many threads

Works:

/ffmpeg -v 9 -loglevel 99 -y -r 23.976 -i input_file.mp4 -threads 32 -c:v dnxhd -b:v 36M -pix_fmt:v yuv422p -s:v 1920x1080 -an output_file.mxf

Fails:

/ffmpeg -v 9 -loglevel 99 -y -r 23.976 -i input_file.mp4 -threads 33 -c:v dnxhd -b:v 36M -pix_fmt:v yuv422p -s:v 1920x1080 -an output_file.mxf

comment:6 Changed 2 months ago by richardpl

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.