unexpected restart of server with ffmpeg 3.x/4.x

[transcoder]

Summary of the bug:
How to reproduce:

        /usr/local/bin/ffmpeg-4.1 -i input.mp4 -codec:a aac -ac 2 -b:a 128k \
        -threads 0 -codec:v h264_nvenc -b:v 3800k -minrate 2400k -maxrate 4500k -bufsize 3800k -flags +loop+mv4 -cmp 256 \
        -partitions +parti4x4+parti8x8+partp4x4+partp8x8+partb8x8 \
        -me_method hex -subq 9 -trellis 1 -refs 3 -bf 3 -deinterlace \
        -x264opts "b-pyramid=normal:weightb:mixed_refs:8x8dct" -coder 1 -me_range 24 \
        -g 25 -keyint_min 25 -r 25 -sc_threshold 0 -i_qfactor 0.71 -qmin 0 \
        -qmax 40 -qdiff 4 -s 1280x720 -aspect 16:9 \
        -psy 1 -fast-pskip 1 \
        -pix_fmt yuv420p  -movflags +faststart -y output.mp4

In the very beginning we had a vod transcoding system with old version ffmpeg
ffmpeg -version
ffmpeg version 0.10.2
built on Apr 27 2012 15:54:07 with gcc 4.4.6 20110731 (Red Hat 4.4.6-3)
configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --mandir=/usr/share/man --enable-runtime-cpudetect --enable-gpl --enable-version3 --enable-nonfree --enable-postproc --enable-avfilter --enable-pthreads --enable-x11grab --enable-vdpau --disable-avisynth --enable-libdc1394 --enable-libdirac --enable-libfaac --enable-libgsm --enable-libmp3lame --enable-libnut --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --extra-cflags='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fPIC' --disable-stripping
libavutil 51. 35.100 / 51. 35.100
libavcodec 53. 61.100 / 53. 61.100
libavformat 53. 32.100 / 53. 32.100
libavdevice 53. 4.100 / 53. 4.100
libavfilter 2. 61.100 / 2. 61.100
libswscale 2. 1.100 / 2. 1.100
libswresample 0. 6.100 / 0. 6.100
libpostproc 52. 0.100 / 52. 0.100

After a while we needed new functions and error resolution (when we did several ss cuts and concat them, there was an error which was not appeared in new 4.1 version).
So we installed a new version
ffmpeg-4.1 -version
ffmpeg version 4.1 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 4.4.7 (GCC) 20120313 (Red Hat 4.4.7-23)
configuration: --prefix=/opt/ffmpeg-4.1/ --extra-cflags=-I/opt/ffmpeg-4.1/include --extra-ldflags='-L/opt/ffmpeg-4.1/lib -ldl' --pkg-config-flags=--static --enable-gpl --enable-libx265 --extra-libs=-lpthread --enable-libfdk_aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-nonfree
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100

After a while (about a week) we faced a problem, server rebooted while transcoding big (about 12GB, same happens after and with less sizes, so it seems does not matter) video with Hd resolution on output. The same video rebooted next server. On a third one it wokrded normally.
So problem may appera once a wee, or twice a day. Before ffmpeg update we never seen before this. Checked everything, memory/disks, updated to latest Centos 6 kernel at that moment (2.6.32-754.10.1.el6) and updalted microcode_ctl.
Did not helped at all. The same reboots still happens. In strace of ffmpeg process we see as just process of encoding happens and stops at some moment (because server rebooted). In dmesg/mcelog nothing.

Initially we thought it was because of centos6. We installed another 2 server with fully updated centos 7 and with ffmpeg
ffmpeg -version
ffmpeg version 3.4.5 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-36)
configuration: --prefix=/var/tmp/ffmpeg-3.4/ --extra-cflags=-I/var/tmp/ffmpeg-3.4/include --extra-ldflags='-L/var/tmp/ffmpeg-3.4/lib -ldl' --pkg-config-flags=--static --enable-gpl --enable-nonfree --enable-libx265 --extra-libs=-lpthread --enable-libfdk_aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-avfilter --enable-libtheora --enable-libvorbis --enable-libopenjpeg
libavutil 55. 78.100 / 55. 78.100
libavcodec 57.107.100 / 57.107.100
libavformat 57. 83.100 / 57. 83.100
libavdevice 57. 10.100 / 57. 10.100
libavfilter 6.107.100 / 6.107.100
libswscale 4. 8.100 / 4. 8.100
libswresample 2. 9.100 / 2. 9.100
libpostproc 54. 7.100 / 54. 7.100

and on another sever

ffmpeg -version
ffmpeg version 4.1 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-36)
configuration: --prefix=/var/tmp/ffmpeg-4.1/ --extra-cflags=-I/var/tmp/ffmpeg-4.1/include --extra-ldflags='-L/var/tmp/ffmpeg-4.1/lib -ldl' --pkg-config-flags=--static --enable-gpl --enable-libx265 --extra-libs=-lpthread --enable-libfdk_aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-nonfree --enable-avfilter --enable-libtheora --enable-libvorbis --enable-libopenjpeg
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100

The same unexpectedy reboots happens on them too. So it happens not only 4.1, and also on 3.4.5 version, even on centos-7

Anybody else faced the same problem? What would you recomend to do in this situation, turn on some debug output? What info should i provide?

[Carl Eugen Hoyos]

Even if there is a bug in FFmpeg (this is not unlikely but nothing in your report explains it), this must never cause the operating system to reboot.
Note that only current FFmpeg git head is supported on this bug tracker.

[transcoder]

"but nothing in your report explains it" - why is that? It began to happen when we compiled new ffmpeg (nothing else changed). Reboots happens only while ffmpeg working.

"Note that only current FFmpeg git head is supported on this bug tracker." - we used release 4.1 version (we compiled it in november 2018). Example with 3.4 version just showed that 3.x version have the same issue.

[Carl Eugen Hoyos]

Replying to transcoder:

"but nothing in your report explains it" - why is that? It began to happen when we compiled new ffmpeg (nothing else changed). Reboots happens only while ffmpeg working.

But reboots are not an indication of a bug in a user application but of a bug in your operating system (or hardware).

"Note that only current FFmpeg git head is supported on this bug tracker." - we used release 4.1 version (we compiled it in november 2018).

But only current FFmpeg git head is supported here, there is no release support.

[transcoder]

"there is no release support" - where should i write about release verions then?

"But reboots are not an indication of a bug in a user application but of a bug in your operating system (or hardware)." - But this bug began to happen ONLY AFTER we compliled new ffmpeg. Also it is 8 servers and at the same time appeared bug in hardware? Not likely. If OS, but centos 6 and centos 7 have the same issues. There is a possibility that it is OS, but again, that means that new ffmpeg began to use something, that leads to reboots, old ffmpeg did not this.
How to dig deeper to find out what is the reason? What can you recommend?

[Carl Eugen Hoyos]

You should ask for release support whoever provided your binaries.

[transcoder]

Where is this release support?