Introduction

This page shows the results from the student projects that were part of GSoC 2024.

---

Projects

Enhancement of FFmpeg's libvmaf Filter ^{(Yigithan Yigit)}

My project was adding a public API to libmvaf(vmaf) to allow per-frame metadata propagation. This explanation might be little abstract let me explain;

Libvmaf can make feature extractions and fuse those extracted features to calculate/predict model feature(s) (vmaf itself is a predicted). You can see that extracted features by setting output file in vmaf cli or FFmpeg, but problem is you can only see that file when process is finished, in other words you can’t see extracted features while doing feature extraction or prediction.

My project was propagate those features to a user defined callback function when features are ready. In short I made program able to show extracted features when they extracted instead of waiting to process finish.

Implementing the eSSIM metric ^{(Andrea Mastroberti)}

The goal was to implement options for the vf_ssim.c filter in libavfilter such that it could compute eSSIM (enhanced SSIM). To do this, I've implemented options for integral images, resizable window size/stride for local statistics and pooling methods to aggregate said statistics (in particular, Minkowski pooling).

​https://gist.github.com/dre-droid/effd22ce0f7ed13914d446367b4bab64

x86 assembly code for the VVC decoder ^{(Stone Chen)}

The goal was to implement x86 assembly code for the VVC decoder, focusing on two major functions: the DMVR SAD and Deblocking. Stone contributed by upstreaming the SAD function, achieving a performance boost of approximately 16-25x for SAD itself. The Deblocking function is nearly complete, with only the Luma long filter remaining. He will continue working on this until it is fully upstreamed, sharing much of the code with HEVC.

Updating FFmpeg's SITI Video Filter to Support HDR and ITU-T P.910:2022 Compliance ^{(Poorva Gaikar)}

My project focused on enhancing FFmpeg's SITI (Spatial Information and Temporal Information) filter to ensure compliance with the latest ITU-T P.910 specification, specifically the 07/22 revision. This involved updating the calculation methods to support high dynamic range (HDR) content.

The SITI filter measures spatial and temporal distortions in video, and my work included refining the Spatial Information (SI) and Temporal Information (TI) calculations for greater accuracy. To achieve this, I integrated the VQEG SITI-tools' Python reference implementation into the C-based FFmpeg codebase, ensuring consistency between both implementations.

In addition to ensuring compliance with ITU-T P.910, I also addressed issues related to the scaling and normalization of luma values, which are crucial for processing HDR content.