ffmpeg is triggering an assertion and the assertion is causing a malloc corruption

[crow]

Summary of the bug: it looks like ffmpeg is triggering an assertion (which it shouldn't) and the assertion is causing a malloc corruption. The malloc corruption probably isn't a bug in ffmpeg (but in libc), but the assertion triggering seems like a probable bug in ffmpeg.
This above was written by "c_14" on IRC!

The backtrace could be found in the attachment.

This isn't happening with the ffmpeg 3.2.4 .

How to reproduce: with using VDR and its vdr-softhdevice output plugin with Live TV. I'll try to provide recording of this Live TV channel with the ffmpeg 3.2.4 and provide it here.

% ffmpeg -i input ... output
ffmpeg version 3.3
built on Archlinux x86_64 with glibc 2.25

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

[crow]

backtrace from crash

[Cigaes]

It is hard to say who is responsible for what here. A few issues are yet obvious:

• the libc is calling i18n functions from an assert failure handler, that is just stupid;

• the crashing code is using assert() instead of av_assert0().

Now, neither of these is the cause of the bug itself. There are two possibilities:

• either this is really a bug in FFmpeg;

• or vdr is somehow corrupting the memory, and that later triggers the assertion failure.

Anyway, it is not a part of the code that I know.

[Carl Eugen Hoyos]

Please run a bisect to find out which change triggers the regression.

[crow]

I tried bisect but there were two commits which could not be build here so i skip them. Please not that compile of 12 commits were successful so I can only guess it is not the build system.

bisect
good n3.2.4
bad n3.3 and current master as of today

$ git bisect reset
We are not bisecting.
$ git bisect start
$ git bisect good d4b731e271ba944ade8f6a128271479529507de9
$ git bisect bad 1968a1eef1cae22e162259d7082c2eea98d81e32
Bisecting: 930 revisions left to test after this (roughly 10 steps)
[928db5134478b4a410d9256a4d56cd8a7d4989b9] Merge commit '27085d1b47c3741cc0fac284c916127c4066d049'
$

$ git bisect skip
There are only 'skip'ped commits left to test.
The first bad commit could be any of:
38efff92f1ef81f3de20ff0460ec7b70c253d714
4a9bab3db0d9ec449ebc8b5e823374d1d1df7761
744801989099df26e90b00062c645969c5347533
We cannot bisect more!
$

Why this commits were skipped:

38efff92f1ef81f3de20ff0460ec7b70c253d714
4a9bab3db0d9ec449ebc8b5e823374d1d1df776164

is because i needed to exclude almost half of stuff from ./configure and then again there were such compile errors

Unknown option(s)

--disable-stripping \
--enable-fontconfig \
etc etc.

HOSTCC  doc/print_options.o
In file included from doc/print_options.c:32:0:
./libavcodec/options_table.h:87:1: warning: ‘me_method’ is deprecated [-Wdeprecated-declarations]
 {"me_method", "set motion estimation method", OFFSET(me_method), AV_OPT_TYPE_INT, {.i64 = ME_EPZS }, INT_MIN, INT_MAX, V|E, "me_metho
d"},
 ^
In file included from ./libavformat/avformat.h:258:0,
                 from doc/print_options.c:29:
./libavcodec/avcodec.h:1627:30: note: declared here
     attribute_deprecated int me_method;
                              ^~~~~~~~~
...
...
In file included from ./libavformat/avformat.h:258:0,
                 from doc/print_options.c:29:
./libavcodec/avcodec.h:3035:9: note: declared here
     int side_data_only_packets;
         ^~~~~~~~~~~~~~~~~~~~~~
HOSTLD  doc/print_options
GENTEXI doc/avoptions_format.texi
GENTEXI doc/avoptions_codec.texi
POD     doc/avconv.pod
MAN     doc/avconv.1
POD     doc/avprobe.pod
MAN     doc/avprobe.1
CC      tools/qt-faststart.o
LD      tools/qt-faststart
make: *** No rule to make target 'doc/ffmpeg.1'.  Stop.
==> ERROR: A failure occurred in build().
    Aborting...
==> ERROR: Build failed

[crow]

I was able to compile one skiped commit by disabling building of docs (thanks BtbN from IRC), and marked that as good and got:

744801989099df26e90b00062c645969c5347533 is the first bad commit

​http://git.ffmpeg.org/gitweb/ffmpeg.git/commit/744801989099df26e90b00062c645969c5347533

[crow]

Is something else needed from me regarding this bug report? Below you can find ffprobe from recording which was taken with VDR (vdr-sotfhddevice compiled against ffmpeg 3.2.4) which can be replayed just fine, but replay fails and crash with VDR (vdr-softhddevice compiled against ffmpeg 3.3.)

ffprobe

$ ffprobe -i /srv/vdr/video/test/2017-05-04.19.45.16-0.rec/00001.ts
ffprobe version 3.3 Copyright (c) 2007-2017 the FFmpeg developers
  built with gcc 6.3.1 (GCC) 20170306
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxvid --enable-netcdf --enable-shared --enable-version3
  libavutil      55. 58.100 / 55. 58.100
  libavcodec     57. 89.100 / 57. 89.100
  libavformat    57. 71.100 / 57. 71.100
  libavdevice    57.  6.100 / 57.  6.100
  libavfilter     6. 82.100 /  6. 82.100
  libavresample   3.  5.  0 /  3.  5.  0
  libswscale      4.  6.100 /  4.  6.100
  libswresample   2.  7.100 /  2.  7.100
  libpostproc    54.  5.100 / 54.  5.100
[h264 @ 0x55cae0d8f6e0] mmco: unref short failure
    Last message repeated 1 times
[mpegts @ 0x55cae0d8ac40] PES packet size mismatch
    Last message repeated 1 times
Input #0, mpegts, from '/srv/vdr/video/test/2017-05-04.19.45.16-0.rec/00001.ts':
  Duration: 00:00:58.65, start: 71545.302244, bitrate: 3101 kb/s
  Program 132
    Stream #0:0[0x6ff]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg, top first), 720x576 [SAR 16:11 DAR 20:11], 24.92 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x700](qae): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s
    Stream #0:2[0x701](qaf): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s
$

[crow]

same problem also with ffmpeg 3.3.1

[Hendrik]

You should try to reproduce the problem without any external software, but only with ffmpeg, and then share a sample file that triggers the problem. Debugging problems that require external software to occur complicates the entire process.

Note that you are using the old and deprecated vdpau code, which is scheduled to be removed in the future (and likely not much tested or used anymore), so I would generally recommend to get VDR updated to the newer vdpau API in libavcodec.

[crow]

@heleppkes
Can you give me an example how to test directly by using ffmpeg? Here I have an .ts recording, but I am unable to find right examples how to play this file with ffmpeg directly on my TV (Xorg running).

I'll pass this info regarding deprecated vdpau code to the vdr-softhddevice (output plugin for VDR) developer.

[Balling]

Can we close this?

[Elon Musk]

Feel free to reopen if still happens.