`av_opt_set_int()` overflow/not-working bug

[think3r]

av_opt_set_int() function will overflow/failed when set a large uint64_t number

• below is the test code from libavutil/tests/opt.c :

diff --git a/libavutil/tests/opt.c b/libavutil/tests/opt.c
index 5799e45c6a..c950cfab16 100644
--- a/libavutil/tests/opt.c
+++ b/libavutil/tests/opt.c
@@ -49,6 +49,7 @@ typedef struct TestContext {
     void *binary2;
     int binary_size2;
     int64_t num64;
+    uint64_t numu64;
     float flt;
     double dbl;
     char *escape;
@@ -86,6 +87,7 @@ static const AVOption test_options[]= {
     {"bin1",       "set binary value",   OFFSET(binary1),        AV_OPT_TYPE_BINARY,         { .str=NULL },                     0,         0, 1 },
     {"bin2",       "set binary value",   OFFSET(binary2),        AV_OPT_TYPE_BINARY,         { .str="" },                       0,         0, 1 },
     {"num64",      "set num 64bit",      OFFSET(num64),          AV_OPT_TYPE_INT64,          { .i64 = 1 },                      0,       100, 1 },
+    {"numu64",     "set num u64",        OFFSET(numu64),         AV_OPT_TYPE_UINT64,         { .i64 = 0xb4000078c6bdbb60 },     0,  UINT_MAX, 1 },
     {"flt",        "set float",          OFFSET(flt),            AV_OPT_TYPE_FLOAT,          { .dbl = 1.0 / 3 },                0,       100, 1 },
     {"dbl",        "set double",         OFFSET(dbl),            AV_OPT_TYPE_DOUBLE,         { .dbl = 1.0 / 3 },                0,       100, 1 },
     {"bool1",      "set boolean value",  OFFSET(bool1),          AV_OPT_TYPE_BOOL,           { .i64 = -1 },                    -1,         1, 1 },

• The error : Value -5476376628152124416.000000 for parameter 'numu64' out of range [0 - 4.29497e+09]

• Reason : Loss of precision when convert between int64_t/uint64_t and double. Which will happends on :
  1. AVOption.max and AVOption.min
  2. write_number() in file libavutil/opt.c

[think3r]

I found this bug while fixing the ijkPlayer crash on Android-11

1. ​https://github.com/bilibili/ijkplayer/issues/52060
2. ​https://source.android.com/docs/security/test/tagged-pointers

[quinkblack]

Pass pointer via av_opt_set_int()/av_opt_get_int() isn't a supported usecase.
FFmpeg doesn't do it internally, it's a tweak from a modified version of FFmpeg.
I know how to do the tweak more robust, but it's out of topic.