minterpolate filter does not work on huge video files

[Dmitry]

We are working with virtual reality videos and it is not uncommon for those videos to be bigger than 4096x4095.
Ticket 6795 helped a lot but we have more and more videos in a very high resolution.
Is it possible to remove resolution limits completely from minterpolate filter?
How to reproduce:

% ./ffmpeg [input] -vf minterpolate [output]
ffmpeg version N-45194-g40102a213-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
...
    Stream #0:0(und): Video: hevc (Main) (hev1 / 0x31766568), yuv420p(tv), 6400x6400, 53472 kb/s, 30 fps, 30 tbr, 90k tbn, 29.97 tbc (default)
...
[Parsed_minterpolate_0 @ 0x74d3640] Failed to configure input pad on Parsed_minterpolate_0d=N/A    
Error reinitializing filters!
Failed to inject frame into filter network: Cannot allocate memory
Error while processing the decoded data for stream #0:1
Conversion failed!

[Timo R.]

The error indicates that your system plain ran out of memory.
I cannot see any frame size limitation in the filter, the limit is your RAM.

[Dmitry]

See ticket 6795, cehoyos explained the problem.
And this patch: ​http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=ad73b32d2922f4237405043d19763229aee0e59e
It's not an out of memory problem

[Carl Eugen Hoyos]

Any reason you made it as difficult as possible to reproduce for everybody interested in this ticket?

[Dmitry]

Replying to cehoyos:

Any reason you made it as difficult as possible to reproduce for everybody interested in this ticket?

Yes, this is the same issue as ticket 6795 which was already reported, reproduced, discussed and partially fixed. I thought that it will be a good idea to sponsor this issue complete fix. Sorry for this.

[Elon Musk]

It is ok to report bug reports, do you still want to sponsor complete fix?

[Dmitry]

Replying to richardpl:

It is ok to report bug reports, do you still want to sponsor complete fix?

Yes, please email me ​cto@s3for.me

[Carl Eugen Hoyos]

Please test the following inlined patch with -max_alloc

diff --git a/libavfilter/vf_minterpolate.c b/libavfilter/vf_minterpolate.c
index b0bb238ade..d64f48a548 100644
--- a/libavfilter/vf_minterpolate.c
+++ b/libavfilter/vf_minterpolate.c
@@ -361,9 +361,11 @@ static int config_input(AVFilterLink *inlink)
     }
 
     if (mi_ctx->mi_mode == MI_MODE_MCI) {
-        mi_ctx->pixel_mvs = av_mallocz_array(width * height, sizeof(PixelMVS));
-        mi_ctx->pixel_weights = av_mallocz_array(width * height, sizeof(PixelWeights));
-        mi_ctx->pixel_refs = av_mallocz_array(width * height, sizeof(PixelRefs));
+        if (width * height < SIZE_MAX / 4 / FFMAX3(sizeof(PixelMVS), sizeof(PixelWeights), sizeof(PixelRefs))) {
+            mi_ctx->pixel_mvs = av_mallocz(width * height * sizeof(PixelMVS));
+            mi_ctx->pixel_weights = av_mallocz(width * height * sizeof(PixelWeights));
+            mi_ctx->pixel_refs = av_mallocz(width * height * sizeof(PixelRefs));
+        }
         if (!mi_ctx->pixel_mvs || !mi_ctx->pixel_weights || !mi_ctx->pixel_refs) {
             ret = AVERROR(ENOMEM);
             goto fail;

[Dmitry]

Patch test

[Dmitry]

Replying to cehoyos:

Please test the following inlined patch with -max_alloc

Works!
It took a lot of time and about 32G of memory but it's not a problem. See attachment ffmpeg_interpol_6400x6400.txt

I'll do more testing next week to see how it works.

P.S. Initial sponsor proposition is still valid - ​cto@s3for.me

Thank you!

[Carl Eugen Hoyos]

Another work-around with the same effect was pushed as b7d9507bb8c4d1b8bf99158d6859a5b2ecd73298