wiki:

FATE

/

AddingATest


Version 4 (modified by Timothy_Gu, 5 years ago) (diff)

--

Adding a FATE Test

There are two main kinds of FATE test: using existing samples (primarily useful with decoder testing), and using an artificially generated sample for testing.

The recipes for the tests written in GNU Make are located under tests/fate in the FFmpeg source. Because FFmpeg has a very complex codebase, differing code needs a wide variety of tests. Let us start with the common part of the tests.

Four Parts of a Test

A typical test usually consists of four parts:

  1. Registration: meaning that make fate automatically execute the test you want to add;
  2. Dependencies: which are usually samples for a format, a generated sample, or a program specifically written to test this feature;
  3. Configuration: this part is the most important of all, because it controls how the test will be run.

Let us focus on the separate parts of a test.

Registration

One Test for one Feature

The most common case is that only one test is allocated for one feature (e.g. decoder, muxer, hashing algorithm, etc.). Let's open tests/fate/audio.mak, which contains tests for audio codecs, and scroll down to a test named fate-dts. For convenience, you can also see a copy of the recipe below.

FATE_SAMPLES_AUDIO-$(call DEMDEC, MPEGTS, DCA) += fate-dts
fate-dts: CMD = pcm -i $(TARGET_SAMPLES)/dts/dts.ts
fate-dts: CMP = oneoff
fate-dts: REF = $(SAMPLES)/dts/dts.pcm

This is one of the simplest tests for FFmpeg. Look at the first quoted line. It adds fate-dts to a weird variable containing a call to DEMDEC. The DEMDEC function returns "yes" if the required DEMuxer and DECoder, in this case the mpegts demuxer and dca decoder, are enabled. Then, if you look at the end of the file:

FATE_SAMPLES_AUDIO += $(FATE_SAMPLES_AUDIO-yes)

FATE_SAMPLES_FFMPEG += $(FATE_SAMPLES_AUDIO)

The enabled tests are added into FATE_SAMPLES_AUDIO and subsequently FATE_SAMPLES_FFMPEG variable, which then of course gets called when doing make fate.

Keep in mind that different tests may have different variable prefixes. Not all tests can be categorized into FATE_SAMPLES_FFMPEG variable.

If only one component for the test you want to add is required, you can replace $(call DEMDEC, MPEGTS, DCA) to $(CONFIG_*).

Filter Tests

Decoder Tests

libavformat Tests

libswresample Tests