Adding a FATE Test

There are two main kinds of FATE test: using existing samples (primarily useful with decoder testing), and using an artificially generated sample for testing.

The recipes for the tests written in GNU Make are located under tests/fate in the FFmpeg source. Because FFmpeg has a very complex codebase, differing code needs a wide variety of tests. Let us start with the common part of the tests.

Four Parts of a Test

A typical test usually consists of four parts:

1. Registration: meaning that make fate automatically execute the test you want to add;
2. Dependencies: which are usually samples for a format, a generated sample, or a program specifically written to test this feature;
3. Configuration: this part is the most important of all, because it controls how the test will be run.

Let us focus on the separate parts of a test.

Registration

One Test for one Feature

The most common case is that only one test is allocated for one feature (e.g. decoder, muxer, hashing algorithm, etc.). Let's open tests/fate/audio.mak, which contains tests for audio codecs, and scroll down to a test named fate-dts. For convenience, you can also see a copy of the recipe below.

FATE_SAMPLES_AUDIO-$(call DEMDEC, MPEGTS, DCA) += fate-dts
fate-dts: CMD = pcm -i $(TARGET_SAMPLES)/dts/dts.ts
fate-dts: CMP = oneoff
fate-dts: REF = $(SAMPLES)/dts/dts.pcm

This is one of the simplest tests for FFmpeg. Look at the first quoted line. It adds fate-dts to a weird variable containing a call to DEMDEC. That function returns

Filter Tests

Decoder Tests

libavformat Tests

libswresample Tests