Consider adding a fuzzer for filtergraphs

[somehacker]

Hi!

I have started developing a fuzzer for filtergraphs which you can read about here: ​https://personnumber3377.github.io/projects/fuzzing_ffmpeg_avfilter_graph_parse_ptr.html . I made a fork of ffmpeg here: ​https://github.com/personnumber3377/ffmpeg which has the fuzzer source code. I already found a couple of bugs with it (for example ​https://trac.ffmpeg.org/ticket/11395). I don't know how worth it it is to fuzz filtergraphs, because they shouldn't be attacker controlled anyway since usually the filtergraphs are specified by developers when using ffmpeg in other projects and only the input and output files should be controlled by users (and possible attackers). My fuzzer source code is based on the example code in doc/examples/decode_filter_video.c and doc/examples/decode_filter_audio.c . My fuzzer works by first validating the filtergraph for syntax by calling avfilter_graph_parse_ptr and then continuing if it is valid and then just having a small video file in memory which is then read with fmemopen similar to an actual file and then processed with the fuzzer generated filtergraph string. I am only getting one or two execs a second when running my fuzzer even though the in-memory video file is just a couple of kilobytes.

Maybe someone can optimize it further or maybe do a complete rewrite of this even???

Thanks in advance for your ideas and thoughts!

[somehacker]

Forgot to mention that since the syntax of the filter graphs are highly structured, maybe it would be worth it to create a custom mutator which would always produce syntactically valid filtergraph expressions such that the majority of them don't get rejected at the parsing stage. I think this can be done by programmatically going through the source code and then processing the different filters and their options and adding a handler for each type of option (AV_OPT_TYPE_INT, AV_OPT_TYPE_STRING, AV_OPT_TYPE_FLOAT etc etc). Then adding mutation strategies, like adding options, removing options, mutating option value, adding input pads, removing input pads or output pads and so on. This is basically on the level of "metaprogramming" and quite ambitious but I think it is doable.