Introduction
This page shows the results from the student projects that were part of GSoC 2023.
Projects
VVC Decoder Improvements
Description:
At the time of the project, FFmpeg's native VVC decoder was not yet merged. The objective of the project was to assist in developing the decoder by improving it in two ways: implementing some of the remaining features of the standard and improving its performance by writing SIMD assembly optimisations.
Results:
Support for the higher bit depths enabled by the 04/22 revision of the standard was implemented. This required implementing the Range Extension feature, which enables increasing the precision of stored transform coefficients. This involved changes to a range of the code, including the inverse transforms and CABAC.
In the process of implementing the Range Extension, a bug was discovered in the chroma intra mode derivation. Research revealed this to be a bug not in FFmpeg's decoder but in the standard itself, and so a ticket was raised with the standards group which has resulted in a correction being issued in the 09/23 revision of the standard. This fix was implemented in the decoder.
Future Work:
It was initially planned to adapt AVX-2 optimisations from dav1d for the inverse transforms. This proved impossible however as the lower precision of the transform coefficients in VVC means they do not satisfy the trigonometric identities needed for the algorithm dav1d uses for its inverse transforms. An attempt was then made to port the AVX-2 inverse transform optimisations from FFmpeg's native HEVC decoder however this also ran into issues, due to the large number of transform sizes in VVC. Some work has been done on writing custom AVX-2 transform optimisations specifically for the VVC decoder. These are composable like dav1d's transforms however use 1-D transform algorithms suitable for VVC. Larger sizes need to be implemented before these optimisations can be merged.
Mentor: Nuo Mi
Contributor: Frank Plowman (post@frankplowman.com)
Audio Overlay Filter
Description:
The Audio Overlay Filter was designed as a part of this project to provide the functionality of replacing specified parts of an input stream with another input stream. Additionally, FATE tests were designed for the filter to ensure that it works as expected in the future.
The filter provides two modes of operation: timeline mode in which the second stream is output in place of the first stream at timestamps provided by the user, and default mode in case no time interval is given by the user, in which the second stream is output at places in the first stream where there are PTS gaps.
At points of transition between the two streams, the filter performs linear crossfading to ensure smooth transitions. The duration for the crossfade can be specified by the user.
The FATE tests are designed to confirm both the modes of operation (with and without the default crossfade duration) work correctly.
Results:
The project was successfully completed and the code along with the tests is currently under review on Patchwork: https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10502
A more elaborate explanation of the filter's functionality can be found on this GitHub gist: https://gist.github.com/HarshK23/703946316d98f5f3bf4e0442f956bd7d
Future Work:
The capabilities of the filter can be further extended by implementing support for more types of crossfading at stream transitions, similar to the acrossfade filter.
Mentors:
- Paul B Mahol
- Thilo Borgmann
Contributor: Harshit Karwal (karwalharshit@gmail.com)
