wiki:

SponsoringPrograms

/

GSoC

/

2021

Introduction

FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter, and stream audio and video. It is used by a number of media players (including VLC and MPlayer), video websites (including YouTube and Vimeo), web browsers (including Google Chrome/Chromium and Firefox), social media (including Facebook and Twitter) and countless other software projects in the multimedia field and beyond.

This is our ideas page for Google Summer of Code 2021. See the GSoC Timeline for important dates.

At the end of the program you can find all the results on the results page.


Information for Students

Getting Started

  1. Get to know FFmpeg. If you are a student interested in contributing to FFmpeg, it is recommended to start by subscribing to the ffmpeg-devel mailing-list, visiting our FreeNode IRC channels (#ffmpeg-devel and #ffmpeg although you will find most developers on the first one), and exploring both the codebase and the development workflow. Feel free to contact us if you have any questions. Also do not hesitate to answer questions from other students on our IRC channel if you know the answer to something.
  1. Find a project. Listed on this page are mentored and un-mentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Un-mentored projects are additional ideas you may want to consider, but you will have to contact us to find a mentor. You can also propose your own project, if you can think of one that better fits your interest and skill level. If a project description is unclear or you have any questions, please get in touch with its mentor and/or join our IRC channel at #ffmpeg-devel.
  1. Contact us. If you decide on a project, get in touch with the community and let us know. If you want to work on a qualification task, let the respective mentor know so we can avoid duplicated efforts.
  1. Apply. Students should apply definitely before deadline on April 13th. The "work" period begins on June 7th and ends in August. Take a look at GSoC timeline for additional information. Note, make sure you apply to Google before April 13th, even if you have not yet finished your qualification task. Please apply as soon as possible: Applications can be improved until the 13th but not afterwards!

Note: A friendly reminder that while the application to GSoC is important for you and GSoC, FFmpeg mentors will not base their decision solely on the GSoC application. We will judge applicants based on their qualification tasks to understand their abilities in coding, learning the tools, communication skills etc. So please do not worry about your application being perfect for us. Although it is very important to follow GSoC's application rules so they can pay you.

Qualification Tasks

In order to get accepted you have to complete a small qualification task which in all cases include sending a patch to the development mailing list. FFmpeg development can be quite challenging and the qualification task helps us figure out whether you are motivated enough and have the potential to deliver successfully.

The qualification tasks are usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the FFmpeg Bug Tracker for qualification task ideas. In general qualification tasks should include submitting a patch to the ffmpeg-devel mailing list which passes review and is accepted into the FFmpeg codebase. It will be common for such patches to need multiple iterations of submissions and reviews, so don't wait too long with the first submission! Note, please avoid picking a qualification task which another student is already working on, each student should work on a different qualification task.

Development

If you are selected for a particular project then you are not only expected to present a working implementation but you should also submit your work for inclusion for the ffmpeg codebase. This should be done at least 2-3 weeks before the end of the second work period by sending patches to the ffmpeg-devel mailing list where the ffmpeg community and your mentor will review your work. You will likely be asked to make some changes and resend improved versions. If you feel that no consensus is reached about how something should be done then follow the advice of your mentor.

In order to create good quality patches make sure to read the Developer Documentation.

Contacting FFmpeg

If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:

  • Mailing-list: ffmpeg-devel
  • IRC: #ffmpeg-devel on Freenode
  • FFmpeg GSoC Admins: Michael Niedermayer (michaelni in #ffmpeg-devel on Freenode IRC, michaelni@gmx.at), Reynaldo Verdejo (reynaldo in #ffmpeg-devel on Freenode IRC, R Verdejo on g mail), Carl Eugen Hoyos (cehoyos in #ffmpeg-devel on Freenode IRC, ceffmpeg at gmail dot com), Thilo Borgmann (thilo.borgmann at mail dot de)

You may also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.


Mentored Projects

This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.

Template Project

Description: Template description.

Expected results: Template expected results.

Prerequisites: Template prequisites.

Qualification Task: Template qualification task.

Mentor: Template (template [at] gmail [dot] com)

Backup Mentor: Template (template [at] gmail [dot] com)

Fuzzing FATE tests in libavformat, libavutil and libswscale

Description: FATE is FFmpegs self / regression test system. Fuzzing with oss-fuzz is used to find corner cases which trigger anomalies like undefined behavior. FATE tests are sub optimal to be part of fuzzing as they use hardcoded parameters as input for each test. For example libavutil/tests/des.c runs some random and fixed tests but no tests controlled by some data from a fuzzer. So even if a anomaly occurs during a random test, this would not be reproducible nor could a fuzzer direct execution towards more anomalous cases.

Expected results: Connect about 1/3 to half, more if time permits of libavformat/tests/*.c libavutil/tests/*.c libswscale/tests/*.c tests to oss-fuzz so that the fuzzer can fully affect the data in each test.

Prerequisites: Good C and Makefile coding skills, basic familiarity with Git, knowledge of oss-fuzz.

Qualification Task: Connect 2 of the 60 FATE tests from libavutil / libavformat / libswscale to work under oss-fuzz, one from libavutil and one from libavformat. Passing qualification will be judged by having working code pushed into FFmpeg git and oss-fuzz git before the deadline given to us for choosing projects.

Mentor: Michael Niedermayer (michaelni in #ffmpeg-devel on Freenode IRC, michaelni@gmx.at)

Backup Mentor: Carl Eugen Hoyos (ceffmpeg [at] gmail [dot] com)

Fuzzing FATE tests in libavcodec, libavfilter and libswresample

Description: FATE is FFmpegs self / regression test system. Fuzzing with oss-fuzz is used to find corner cases which trigger anomalies like undefined behavior. FATE tests are sub optimal to be part of fuzzing as they use hardcoded parameters as input for each test. For example libavcodec/tests/snowenc.c runs some random and fixed tests but no tests controlled by some data from a fuzzer. So even if a anomaly occurs during a random test, this would not be reproducible nor could a fuzzer direct execution towards more anomalous cases.

Expected results: Connect about 1/3 to half, more if time permits of libavcodec/tests/*.c libavfilter/tests/*.c libswresample/tests/*.c src/tests/*.c tools/*.c tests/tools to oss-fuzz so that the fuzzer can fully affect the data in each test/tool.

Prerequisites: Good C and Makefile coding skills, basic familiarity with Git, knowledge of oss-fuzz.

Qualification Task: Connect 2 of the 59 FATE tests and tools from libavcodec / libavfilter / libswresample / src / tools to work under oss-fuzz, one from libavcodec and one from libavfilter. Passing qualification will be judged by having working code pushed into FFmpeg git and oss-fuzz git before the deadline given to us for choosing projects.

Mentor: Michael Niedermayer (michaelni in #ffmpeg-devel on Freenode IRC, michaelni@gmx.at)

Backup Mentor: Carl Eugen Hoyos (ceffmpeg [at] gmail [dot] com)

Guided filter

Description: Guided filter is a widely known tool which shows remarkable performance for image defogging and edge-preserving filtering. The student should develop an effective and efficient guided filter in ffmpeg.

Expected results: A working guided filter.

Prerequisites: Good C and Makefile coding skills, basic familiarity with Git, knowledge of machine learning and deep learning.

Qualification Task: Passing qualification will be judged by having working code for guided filter pushed into FFmpeg git.

Mentor: Steven Liu (liuqi05 [at] kuaishou [dot] com)

Backup Mentor: Jun Zhao (mypopy [at] gmail [dot] com)

H264/HEVC ARM64 10-bit assembly

Description: Modern codecs require handwritten SIMD assembly for good performance, much of this has been implemented for 8-bit depth in h264 and hevc but there are still missing for 10 (h264) and 10/12 (hevc).

Expected results: Full or greatly increased coverage for either 10bit h264 or 10/12bit hevc decode assembly.

Prerequisites: Good C skills, basic ARM/64 assembly knowledge and basic familiarity with Git.

Qualification Task: Implement a single 10-bit function.

Mentor: Josh Dekker (josh [at] itanimul.li)

Backup Mentor: Thilo Borgmann (thilo.borgmann [at] mail.de)

FFT assembly

Description: libavutil has a new, modern frequency transformation infrastructure. While the most important architectures (x86 and ARM) and features (Float FFT and MDCT) are covered or work in progress, there's still a lot missing, such as 64-bit float versions of FFTs and MDCTs for x86, ARM and RISC-V, as well as Float, Integer and Double optimizations for PowerPC. It's up to you to decide which version and architecture to pick.

Expected results: Complete or mostly done FFT and MDCT optimizations for your chosen version(s) and architecture(s)

Prerequisites: Decent assembly knowledge in your chosen architecture, some C skills, and basic familiarity with Git.

Qualification Task: Implement a 4-point FFT based on this unrolling.

Mentor: Lynne (Lynne on Freenode IRC or @lynne:pars.ee on Matrix)

Backup Mentor: Paul B Mahol (durandal_1707 on Freenode IRC or onemda [at] gmail [dot] com)


Unmentored Projects

This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.

Template Project

Description: Template description.

Expected results: Template expected results.

Prerequisites: Template prequisites.

Qualification Task: Template qualification task.


Your Own Project Idea

A student can also propose a self-defined project! The amount of work defined by such a project should last for the majority of the GSoC work period duration and the task must be approved by the community. Also, a capable mentor from the community must be assigned. Ideas and inspiration for such project ideas could be found by browsing bugs and feature requests on our bug tracker but are of course not limited to such.

Students can discuss an idea with the community via the ffmpeg-devel mailing-list, the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins directly for more information. It is obviously of importance to start such a discussion well before the application deadline.

Last modified 4 hours ago Last modified on Feb 27, 2021, 8:15:08 PM