Background

The ​Sovereign Tech Fund invests globally in the open software components that underpin Germany's and Europe's competitiveness and ability to innovate.

As summarized on their web page:

The Sovereign Tech Fund invests in open digital base technologies that are vital to the development of other software or enable digital networking. We invest in projects that benefit and strengthen the open source ecosystem. Examples include libraries for programming languages, package managers, open implementations of communication protocols, administration tools for developers, digital encryption technologies, and more. We do not finance the development of prototypes. Our currently funded projects show the range and type of technologies that characterize open digital base technologies. [The Sovereign Tech Fund is] currently not looking for user-facing applications, such as messaging apps or file storage services.

The FFMPEG community secured ​STF funding activities in 2024 (report) and plans to do so again this year.

If you are interested in being included in the application and believe your project fits the criteria above, please email the information specified in the template below to:

• Michael Niedermayer (​michael-stf@niedermayer.cc)
• Thilo Borgmann (​thilo.borgmann@mail.de)
• Pierre-Anthony Lemieux (​pal@sandflow.com)

The deadline to submit proposals is September 1, 2025.

Each developer whose proposal is accepted will need to enter into an agreement with ​Software in the Public Interest (SPI) who will manage the disbursement of STF monies (if awarded). The aggregate value of the SOWs need to be greater than €150,000 (minimum STF funding).

Template

One line summary of the proposed work

Example:

Classify and fix outstanding issues identified by Coverity

Description of the work

Example:

Coverity is a static code analysis system that is used to analyze FFmpeg code to find bugs with an emphasis on quality and security issues. There are currently 677 outstanding issues identified by Coverity (​https://scan.coverity.com/projects/ffmpeg?tab=overview). Some of these issues are false positives while others could open the door to security vulnerabilities. The objective of this work is to identify the Coverity issues that are not false positives, and fix as many as possible.

Milestones

Milestone 1

Description

Example:

Review all outstanding Coverity issues and, for each one, determine whether it is a false positive.

Deliverables

Example:

List of both false positive and real issues posted to the FFmpeg dev mailing list.

Requested compensation (in USD or Euros)

Milestone 2

Etc.

Developer background and contact information

Example:

Michael Niedermayer <​michael-ffwork@niedermayer.cc>. I work in Austria, and have been an active contributor to FFmpeg since 2001 – 22308 commits so far. My work on FFmpeg is regularly supported by third parties and I am one of the founders of ​https://fflabs.eu. I am also extremely familiar with Coverity: I have fixed 690 issues out of 847 Coverity issues fixed in the past. I fixed over 2000 issues found by ossfuzz.

FFMPEG 2025 STF Application

Describe the technology this effort will support.

The proposed effort aims to support FFmpeg, a leading multimedia framework recognized for its versatile capabilities.

FFmpeg serves as a comprehensive solution for decoding, encoding, transcoding, multiplexing, demultiplexing, streaming, filtering, and playback of a wide array of multimedia formats. It stands out by seamlessly handling formats from ancient, obscure standards to cutting-edge ones, making it a cornerstone in the multimedia processing landscape.

Where is this technology being used? Why is it relevant and critical? How does this technology serve the public interest?

FFmpeg is a ubiquitous technology, deeply embedded in the fabric of modern multimedia processing and touching the lives of millions every day. FFmpeg is an integral component in a vast array of end-user devices, including desktops, laptops, mobile phones, smartwatches, and smart TVs. FFmpeg also powers major video distribution and streaming platforms like YouTube, Vimeo, AWS, and Microsoft Azure, shaping the landscape of online content consumption.

The technology extends its influence to TV broadcasts over IP, enabling seamless delivery of multimedia content to a global audience. Beyond mainstream applications, FFmpeg is employed in diverse sectors, from archival projects to the Mars Perseverance Rover, showcasing its adaptability and reliability.

The relevance and criticality of FFmpeg lies in its role as the backbone of multimedia processing. It serves the public interest by facilitating the seamless functioning of everyday technologies, from video playback in web browsers like Firefox and Chromium, to applications like Kodi, MPlayer, and OBS Studio. Furthermore, FFmpeg plays a crucial role in public sector use cases, website interactions, and even in the transmission of multimedia content within emails. In essence, FFmpeg's impact is far-reaching, touching both the private and public spheres. FFmpeg's reliability is paramount for the secure and efficient functioning of digital multimedia.

What are the problems the proposed work is trying to address? What activities do you propose and how do they address the problems described? How do those activities fit into [STF’s mission](​https://www.sovereigntechfund.de/mission)?

The proposed work addresses critical challenges in maintaining FFmpeg's sustainability, security, and innovation. Activities include:

• Security Fixes and Hardening: Addressing bugs and enhancing security through improvements of the fuzzing system.
• Administration of FFmpeg Infrastructure: Ensuring robust infrastructure management for reliability and performance.
• Improvements in Codecs, Formats, and Filters: Enhancing existing implementations.

These activities align with STF’s mission by supporting the maintenance and security of open source components and the strengthening the Open Source ecosystem.

Who is going to be performing the work and how are they qualified to do so?

The work will be performed by key FFmpeg project contributors and other experienced developers. Their qualifications have been demonstrated through years of active involvement in FFmpeg development, participating in diverse tasks including code contributions, code review, user support, and administration.

How is the technology maintained or governed? Is there a community behind the technology, and do they approve of the work?

FFmpeg's maintenance and governance in based on a flat hierarchy, with decisions being made transparently through written communications in mailing lists. The FFmpeg project boasts a robust community of administrators, maintainers, and contributors who actively engage in the decision-making processes through consensus-driven discussions. The proposed work aligns with the community's focus on quality, security, and openness and will/has been approved following the community's operating governance processes. (Note that new tasks were added to the wiki 1 hour before submission).

Describe the projects

FFv1 Vulkan improvements

Description: The Vulkan-based GPU implementation of the FFv1 codec has allowed users to quickly encode and decode on consumer GPUs. However, FFv1 codec continues to be developed, with new features being added. Recently, color remapping and floating-point support (16-bit and 32-bit) was added to version 4 of the codec. The GPU implementation of the codec does not yet support either. This project will ensure the GPU implementation remains in sync with the CPU, and that optimizations will be ported between both implementations.

Expected results: FFv1's GPU implementation will implement the color lookup table, 16-bit floating-point video, 32-bit floating-point video, and where possible, optimizations will be back-ported to the CPU implementation.

Duration: 8 Months

Payment: 30,000 EUR

Developer: Lynne

Milestones

16-bit floating-point video decoding

• Description: Minimum viable implementation of floating-point decoding in the Vulkan decoder. This would allow for the FFv1 Vulkan decoder to be used in professional cinema/CGI productions, where linear 16-bit floating point video is used during pre-production.

• Deliverables: Patches to implement compliant support for it posted on the mailing list.

16-bit floating-point video encoding

• Description: Same as the previous milestone, for the encoder. Minimum viable implementation of floating-point decoding in the Vulkan encoder. This would allow for the FFv1 Vulkan encoder to be used in professional cinema/CGI productions.

• Deliverables: Patches to implement compliant support for it posted on the mailing list.

32-bit floating-point video decoding

• Description: More advanced version, built upon the previous work, to allow the decoder to correctly decode 32-bit floating point video. More and more production is switching to 32-bit video, as 16-bit linear video can reach limits with high dynamic range video.

• Deliverables: Patches to implement compliant support for it posted on the mailing list.

32-bit floating-point video encoding

• Description: Same as the previous milestone, for the encoder. 32-bit floating point video encoding support would be added to the encoder.

• Deliverables: Patches to implement compliant support for it posted on the mailing list.

FFv1 Bayer coding support

Description: FFv1 is an important standard for archival. However, it is capable of more than just long-term compression. The recent GPU implementation has demonstrated that its sufficiently fast to be used as a RAW format, allowing NLEs to work in a fully lossless flow. This project will implement support for common Bayer formats, both for encoding and decoding, along with metadata, allowing FFv1 to be used for professional production.

Expected results: FFv1 is able to encode and decode Bayer pixel ordering, both on the GPU and CPU implementations, with the necessary metadata information carried to permit accurate color reconstruction and flexibility when color grading and mastering.

Duration: 6 Months

Payment: 20,000 EUR

Developer: Lynne

Milestones

FFv1 Bayer video encoding and decoding

• Description: Minimum implementation of Bayer encoding and decoding for the CPU encoder. Both have to be added at the same time as this is adding new features to the codec.

• Deliverables: Patches to implement compliant support for it posted on the mailing list.

FFv1 Vulkan Bayer video encoding and decoding

• Description: Implementation of Bayer encoding and decoding for the Vulkan FFv1 encoder and decoders.

• Deliverables: Patches to implement compliant support for it posted on the mailing list.

swscale Vulkan backend

Description: The recent swscale rewrite has transformed it into a flexible and reliable framework to handle scaling, with the logic being decoupled from the DSP code. This simplifies writing backends with all the benefits and deterministic behaviour of the CPU version. This project will implement a Vulkan backend for the new swscale framework, writing SPIR-V code directly with no assembler, which will allow Vulkan frames (and frames imported from other GPU APIs into Vulkan) to be scaled.

Expected results: swscale Vulkan backend, with feature parity to the C backend

Duration: 12 Months

Payment: 40,000 EUR

Developer: Lynne

swscale AArch64 NEON backend

One line summary: This project will implement an AArch64 NEON backend for the new swscale framework.

Description: The recent swscale rewrite has transformed it into a flexible and reliable framework to handle scaling, with the logic being decoupled from the DSP code. This simplifies writing backends with all the benefits and deterministic behaviour of the reference C implementation. This project will extend the new swscale framework by implementing an AArch64 NEON backend, ensuring that ARM-based platforms (including mobile devices, servers, and single-board computers) benefit from the same deterministic and optimized pipeline as x86_64. This backend will be used to generate optimized NEON SIMD assembly code at runtime for platforms where just-in-time code execution (JIT) is supported, and it will also be used to generate static (non-JIT) assembly files that match the Continuation-Passing Style (CPS) code of the reference C and x86_64 implementations.

Expected results: swscale AArch64 NEON backend, both JIT and non-JIT, with feature parity to the reference C and x86_64 implementations.

Duration: 6 Months

Payment: 30,000 EUR

Developer: Ramiro Polla <​ramiro.polla@gmail.com>

Milestones

asmjit validation backend

• Description: A JIT validation backend will be made using the asmjit library. This will allow us to validate the generated assembly code and obtain performance metrics.

• Deliverables: An initial asmjit JIT validation backend in a separate git repository. An RFC with a performance report posted to the FFmpeg-devel mailing list.

Assembly generation abstraction layer

• Description: The asmjit validation backend will be split in two parts: an abstract assembly generation layer (with no asmjit dependency), and the asmjit code generator based on the abstraction layer. This assembly generation layer should include a register allocator. It will then be possible to add other assembly output generators.

• Deliverables: An update on the RFC on the FFmpeg-devel mailing list with the code split.

Static non-JIT backend

• Description: Using the assembly abstraction layer and the AArch64 NEON SIMD code from the asmjit validation backend, it will be possible to generate static assembly files that match the Continuation-Passing Style code of the reference C and x86_64 implementations. A tool (program or script) will be created to generate the static assembly files. This tool will be integrated into the FFmpeg codebase, and the static assembly files will be generated either at build-time or integrated into the FFmpeg codebase as well (to be determined).

• Deliverables: Series merged to git master, effectively providing an initial AArch64 NEON backend for the new swscale framework.

Evaluate other JIT implementations

• Description: The asmjit validation backend, after having been used to bootstrap the static non-JIT backend, will be revisited. Other JIT generation backends will be evaluated, with a focus on either finding or creating a framework that can be cleanly integrated with FFmpeg's C codebase. This dependency should be lightweight and easy to integrate into FFmpeg itself, guaranteeing the maintainability of the new codebase.

• Deliverables: An RFC with a report on the findings posted to the FFmpeg-devel mailing list. The community will help select the best implementation.

Final JIT backend

• Description: The final AArch64 NEON JIT backend will be integrated into the FFmpeg codebase, based on the implementation selected by the community. Support will be added incrementally for each platform, only after they are each properly tested an validated.

• Deliverables: A pull request to be reviewed by the FFmpeg community.

Fixes of fuzzer found bugs

One Line Summary: Fixing issues found by fuzzers

Description: FFmpeg is tested by more and more automated systems, fuzzers, AI and soon a bug bounty programs. This task is to fund some of the efforts to triage and fix the stream of incoming issues

Expected results: Triaging and fixing most newly incoming issues

Duration: 8 Months

Payment: 60000€

Developer: Michael Niedermayer (​michael-stf@niedermayer.cc)

libswscale modernization (part 2)

One line summary: Port remaining "legacy" code paths from libswscale to the new architecture.

Description: Libswscale is the central image scaling and format conversion component used by the FFmpeg multimedia set of libraries. "Modern" libswscale is the name I have given to the new set of APIs and internal designs (SwsContext, SwsGraph and SwsOps) that were successfully introduced as part of the 2024 "swscale modernization" STF project. The objective of this work is to complete this transition, by porting the remaining code paths (notably, scaling support and color conversion) to the new architecture, thus allowing the "legacy" code paths to be deprecated and phased out of FFmpeg.

Duration: 8 months

Payment: 70.000 EUR

Developer: Niklas Haas <​ffmpeg@haasn.xyz>

Milestones

Develop the SwsOps architecture to support scaling

Description: The existing SwsOps architecture will need to be updated to support scaling operations. This includes necessary R&D and testing to find a design that scales well and does not introduce performance regressions versus the legacy code paths.

Deliverables: An implementation of the necessary new APIs and high-level logic, alongside a reference implementation in C.

Compensation: 20,000 EUR

Updating x86_64 backend

Description: After expanding the SwsOps architecture, the x86_64 backend will need to be updated to support the new scaling operations. This includes implementing optimized versions of the new APIs using SIMD instructions.

Deliverables: An implementation, targeting at least SSE4, AVX2 and AVX-512, of the newly added APIs in Milestone 1.

Compensation: 20,000 EUR

Remaining format coverage

Description: Some, more difficult, formats, which were previously handled by the legacy code paths, are not yet supported by the SwsOps backend. This notably includes 96/128-bit formats, XYZ, Bayer and packed/subsampled formats (e.g. YUYV).

Deliverables: An implementation of the necessary APIs and high-level logic, alongside a reference implementation in C, and possibly an x86_64 code path for the most important formats.

Compensation: 20,000 EUR

Properly integrate CMM layer

Description: Some format conversions currently require conversion between difference colorspaces. This operation is currently handled in a very inefficient manner, as a result of predating the SwsOps design. This layer should be updated to use the new architecture, which will make it vastly more efficient.

Deliverables: The color management layer successfully integrated into the SwsOps design.

Compensation: 10,000 EUR