Changes between Version 12 and Version 13 of FATE/AddingATest

Timestamp:

May 10, 2016, 3:40:25 PM (10 years ago)

Author:

Petru Rares Sincraian

Comment:

add enc_dec_pcm description

FATE/AddingATest

@@ -110 +110 @@
 It is very important for you to declare the dependencies of a test if it uses any external files, like special samples or synthetic files.
 
+=== Configuration ===
+This controls how the test will be run, the parameters of each test and so on. There exist many commands, the basic commands are:
+
+==== enc_dec_pcm ====
+This command first encodes an input file and then decodes the encoded file. It’s useful to test a video encoder. An example of this command is fate-vorbis-encode test, which can be found in `tests/fate/vorbis.mak`. The code of this command is the following:
+
+{{{
+FATE_VORBIS += fate-vorbis-encode
+fate-vorbis-encode: CMD = enc_dec_pcm ogg wav s16le $(TARGET_SAMPLES)/audio-reference/luckynight_2ch_44kHz_s16.wav -c:a vorbis -strict experimental
+fate-vorbis-encode: REF = $(SAMPLES)/audio-reference/luckynight_2ch_44kHz_s16.wav
+fate-vorbis-encode: CMP_SHIFT = 0
+fate-vorbis-encode: CMP_TARGET = 2000
+fate-vorbis-encode: SIZE_TOLERANCE = 12123
+fate-vorbis-encode: FUZZ = 0
+[...]
+fate-vorbis-encode: CMP = stddev
+}}}
+
+* CMD
+  * enc_dec_pcm: indicates the command to be executed.
+  * ogg: indicates the format in which to encode the input file
+  * wav: indicates the format in which to decode the encoded file.
+  * s16le: indicates the codec_name.
+  * file: the input file
+  * @: the other commands to be included
+
+* REF: indicates the reference file, the decoded file will be compared with this file.
+* CMP_SHIFT: to calculate the CMP_SHIFT value you can use the tests/tini_psnr command. A guide how to use it is [http://trac.ffmpeg.org/wiki/FATE/UseOftiny_psnr explained here].
+* CMP_TARGET: it depends on what “CMP=  “ is set. 
+* FUZZ: is the difference that is considered ok between the CMP_TARGET and the actual value.
+* SIZE_TOLERANCE is the difference between the input and output sizes. The value of this parameter is calculated by subtracting the size of the two files.
+
+{{{
+#!div style="border: 1pt dotted; margin: 1em; background-color: #fffff9;"
+
+'''Notes:'''
+
+The CMP_TARGET and FUZZ may differ to one architecture to another. You may need to adjust these values depending on the architecture.
+}}}
+
+To do a new test do the following:
+1. Compute CMP_SHIFT value with tiny_psnr. The program needs the encoder input and decoder output (that is 2 pcm wav or raw files).
+2. Copy the fate test to the .mak file and adjust the CMD options to match your needs.
+3. Run the new test, it should show errors like:
+{{{
+stddev:  296.37 PSNR: 46.89 MAXDIFF: 4810 bytes:  1675800/  1679360
+stddev: |296.37 - 2000| >= 0
+}}}
+4. The CMP_TARGET for your test is 296 and your FUZZ is 0.
+5. The SIZE_TOLERANCE is 1679360 - 1675800 = 3560.
+
+
 == Filter Tests ==