| 215 | | ===== 32-bit floating-point video encoding |
| 216 | | |
| 217 | | * **Description**: Same as the previous milestone, for the encoder. 32-bit floating point video encoding support would be added to the encoder. |
| | 223 | * **Completion improvements**: This milestone will complete the Vulkan implementation's feature parity with the C implementation, so that the new FFv1 features are usable in practical applications, where speed and efficiency matter. |
| | 224 | |
| | 225 | === FFv1 Bayer coding support === |
| | 226 | |
| | 227 | '''Description:''' FFv1 is an important standard for archival. However, it is capable of more than just long-term compression. The recent GPU implementation has demonstrated that its sufficiently fast to be used as a RAW format, allowing NLEs to work in a fully lossless flow. This project will implement support for common Bayer formats, both for encoding and decoding, along with metadata, allowing FFv1 to be used for professional productions. |
| | 228 | |
| | 229 | '''Expected results:''' FFv1 is able to encode and decode Bayer pixel ordering, both on the GPU and CPU implementations, with the necessary metadata information carried to permit accurate color reconstruction and flexibility when color grading and mastering. |
| | 230 | |
| | 231 | '''Duration:''' 4 Months |
| | 232 | |
| | 233 | '''Payment:''' 22,000 EUR |
| | 234 | |
| | 235 | '''Developer:''' Lynne |
| | 236 | |
| | 237 | ==== Developer background and contact information |
| | 238 | |
| | 239 | Lynne <dev@lynne.ee>. Based in Paris, FFmpeg developer for over 7 years and a thousand commits. Has written multiple decoders and encoders for FFmpeg, as well as the entire Vulkan infrastructure. Member of Khronos, developing the Vulkan Video specifications, and the AOM, working on new codecs. |
| | 240 | |
| | 241 | ==== Milestones |
| | 242 | |
| | 243 | ===== FFv1 Bayer video encoding and decoding |
| | 244 | |
| | 245 | * **Description**: Minimum implementation of Bayer encoding and decoding for the CPU encoder. Both have to be added at the same time as this is adding new features to the codec. |
| 223 | | === FFv1 Bayer coding support === |
| 224 | | |
| 225 | | '''Description:''' FFv1 is an important standard for archival. However, it is capable of more than just long-term compression. The recent GPU implementation has demonstrated that its sufficiently fast to be used as a RAW format, allowing NLEs to work in a fully lossless flow. This project will implement support for common Bayer formats, both for encoding and decoding, along with metadata, allowing FFv1 to be used for professional production. |
| 226 | | |
| 227 | | '''Expected results:''' FFv1 is able to encode and decode Bayer pixel ordering, both on the GPU and CPU implementations, with the necessary metadata information carried to permit accurate color reconstruction and flexibility when color grading and mastering. |
| 228 | | |
| 229 | | '''Duration:''' 6 Months |
| 230 | | |
| 231 | | '''Payment:''' 20,000 EUR |
| 232 | | |
| 233 | | '''Developer:''' Lynne |
| | 261 | * **Completion improvements**: Completion of this milestone will feature-complete the Vulkan implementation with the C implementation, enabling FFv1 to be used practically, where compression of camera footage or sensor data is required with realtime or near-realtime performance. |
| | 262 | |
| | 263 | === libswscale AArch64 NEON backend === |
| | 264 | |
| | 265 | **One line summary:** This project will implement an AArch64 NEON backend for the new libswscale architecture. |
| | 266 | |
| | 267 | **Description:** The recent libswscale rewrite is transforming the library into a flexible and reliable architecture to handle scaling, with the logic being decoupled from the DSP code. This simplifies writing backends with all the benefits and deterministic behaviour of the reference C implementation. In order to achieve the goal of deprecating and phasing out the "legacy" code paths from FFmpeg, the new architecture must not introduce performance regressions for platforms with widespread use, such as x86_64 and AArch64, for Linux, macOS, and Windows. This project will extend the new libswscale architecture by implementing an AArch64 NEON backend, ensuring that ARM-based platforms (including mobile devices, servers, and single-board computers) benefit from the same deterministic and optimized pipeline as x86_64. This backend will be used to generate highly optimized NEON SIMD assembly code at runtime for platforms where just-in-time code execution (JIT) is supported, and it will also be used to generate static (non-JIT) assembly files that match the Continuation-Passing Style (CPS) code of the reference C and x86_64 implementations for platforms where JIT is not supported. |
| | 268 | |
| | 269 | **Expected results:** libswscale AArch64 NEON backend, both JIT and non-JIT, with feature parity to the reference C and x86_64 implementations. |
| | 270 | |
| | 271 | **Duration:** 6 Months |
| | 272 | |
| | 273 | **Payment:** 30,000 EUR |
| | 274 | |
| | 275 | **Developer:** Ramiro Polla <ramiro.polla@gmail.com> |
| | 276 | |
| | 277 | I live in Belgium, and have been a contributor to FFmpeg since 2006. I am very familiar with libswscale, both the legacy codebase and the recent modernization project, which I have followed closely. I have been working on AArch64 NEON SIMD optimizations for the past couple of years. Besides contributing to open-source multimedia projects, I have also worked for five years at Hex-Rays, helping develop the IDA Pro reverse engineering software. |
| | 278 | |
| | 279 | ==== Milestones |
| | 280 | |
| | 281 | ===== '''asmjit''' validation backend |
| | 282 | |
| | 283 | * **Description**: A JIT validation backend will be made using the '''asmjit''' library. This will allow us to validate the generated assembly code and obtain performance metrics. |
| | 284 | |
| | 285 | * **Deliverables**: An initial '''asmjit''' JIT validation backend in a separate git repository. An RFC with a performance report posted to the FFmpeg-devel mailing list. |
| | 286 | |
| | 287 | * **Compensation:** 6,000 EUR |
| | 288 | |
| | 289 | ===== Assembly generation abstraction layer |
| | 290 | |
| | 291 | * **Description**: The '''asmjit''' validation backend will be split in two parts: an abstract assembly generation layer (with no '''asmjit''' dependency), and the '''asmjit''' code generator based on the abstraction layer. This assembly generation layer should include a register allocator. It will then be possible to add other assembly output generators. |
| | 292 | |
| | 293 | * **Deliverables**: An update on the RFC on the FFmpeg-devel mailing list with the code split. |
| | 294 | |
| | 295 | * **Compensation:** 6,000 EUR |
| | 296 | |
| | 297 | ===== Static non-JIT backend |
| | 298 | |
| | 299 | * **Description**: Using the assembly abstraction layer and the AArch64 NEON SIMD code from the asmjit validation backend, it will be possible to generate static assembly files that match the CPS code of the reference C and x86_64 implementations, even though they come with a certain overhead compared to the JIT code. A tool (program or script) will be created to generate the static assembly files. This tool will be integrated into the FFmpeg codebase, and the static assembly files will be generated either at build-time or integrated into the FFmpeg codebase as well (to be determined). |
| | 300 | |
| | 301 | * **Deliverables**: Series merged to git master, effectively providing an initial AArch64 NEON backend for the new libswscale architecture. |
| | 302 | |
| | 303 | * **Compensation:** 6,000 EUR |
| | 304 | |
| | 305 | ===== Evaluate other JIT implementations |
| | 306 | |
| | 307 | * **Description**: The '''asmjit''' validation backend, after having been used to bootstrap the static non-JIT backend, will be revisited. Other JIT generation backends will be evaluated, with a focus on either finding or creating a framework that can be cleanly integrated with FFmpeg's C codebase. This dependency should be lightweight and easy to integrate into FFmpeg itself, guaranteeing the maintainability of the new codebase. |
| | 308 | |
| | 309 | * **Deliverables**: An RFC with a report on the findings posted to the FFmpeg-devel mailing list. The community will help select the best implementation. |
| | 310 | |
| | 311 | * **Compensation:** 6,000 EUR |
| | 312 | |
| | 313 | ===== Final JIT backend |
| | 314 | |
| | 315 | * **Description**: The final AArch64 NEON JIT backend will be integrated into the FFmpeg codebase, based on the implementation selected by the community. Support will be added incrementally for each platform (Linux, macOS, Windows), only after they are each properly tested an validated. This JIT implementation will considerably reduce the overhead compared to the non-JIT code, ensuring that there are no performance regressions between the legacy code paths and the new architecture for AArch64 platforms. |
| | 316 | |
| | 317 | * **Deliverables**: A pull request with the final AArch64 NEON JIT backend, is either merged or has received at least one merge approval. |
| | 318 | |
| | 319 | * **Compensation:** 6,000 EUR |
| | 320 | |
| | 321 | === Fixes of fuzzer found bugs === |
| | 322 | |
| | 323 | '''One Line Summary:''' Fixing issues found by fuzzers |
| | 324 | |
| | 325 | '''Description:''' FFmpeg is tested by more and more automated systems, fuzzers, AI and soon a bug bounty programs. This task is to fund some of the efforts to triage and fix the stream of incoming issues |
| | 326 | |
| | 327 | '''Expected results:''' Triaging and fixing most newly incoming issues |
| | 328 | |
| | 329 | '''Duration:''' 9 Months |
| | 330 | |
| | 331 | '''Payment:''' 60000€ |
| | 332 | |
| | 333 | '''Developer:''' Michael Niedermayer (michael-stf@niedermayer.cc) |
| 237 | | Lynne <dev@lynne.ee>. Based in Europe, FFmpeg developer for over 7 years and a thousand commits. Has written multiple decoders and encoders for FFmpeg, as well as the entire Vulkan infrastructure. Member of Khronos, developing the Vulkan specifications. Worked on the AV1 codec specifications. |
| 238 | | |
| 239 | | ==== Milestones |
| 240 | | |
| 241 | | ===== FFv1 Bayer video encoding and decoding |
| 242 | | |
| 243 | | * **Description**: Minimum implementation of Bayer encoding and decoding for the CPU encoder. Both have to be added at the same time as this is adding new features to the codec. |
| 244 | | |
| 245 | | * **Deliverables**: A pull request to implement the feature, passing all tests, is either merged or has received at least one merge approval. |
| 246 | | |
| 247 | | * **Compensation**: 10,000 EUR |
| 248 | | |
| 249 | | ===== FFv1 Vulkan Bayer video encoding and decoding |
| 250 | | |
| 251 | | * **Description**: Implementation of Bayer encoding and decoding for the Vulkan FFv1 encoder and decoders. |
| 252 | | |
| 253 | | * **Deliverables**: A pull request to implement the feature, passing all tests, is either merged or has received at least one merge approval. |
| 254 | | |
| 255 | | * **Compensation**: 10,000 EUR |
| 256 | | |
| 257 | | === libswscale AArch64 NEON backend === |
| 258 | | |
| 259 | | **One line summary:** This project will implement an AArch64 NEON backend for the new libswscale architecture. |
| 260 | | |
| 261 | | **Description:** The recent libswscale rewrite is transforming the library into a flexible and reliable architecture to handle scaling, with the logic being decoupled from the DSP code. This simplifies writing backends with all the benefits and deterministic behaviour of the reference C implementation. In order to achieve the goal of deprecating and phasing out the "legacy" code paths from FFmpeg, the new architecture must not introduce performance regressions for platforms with widespread use, such as x86_64 and AArch64, for Linux, macOS, and Windows. This project will extend the new libswscale architecture by implementing an AArch64 NEON backend, ensuring that ARM-based platforms (including mobile devices, servers, and single-board computers) benefit from the same deterministic and optimized pipeline as x86_64. This backend will be used to generate highly optimized NEON SIMD assembly code at runtime for platforms where just-in-time code execution (JIT) is supported, and it will also be used to generate static (non-JIT) assembly files that match the Continuation-Passing Style (CPS) code of the reference C and x86_64 implementations for platforms where JIT is not supported. |
| 262 | | |
| 263 | | **Expected results:** libswscale AArch64 NEON backend, both JIT and non-JIT, with feature parity to the reference C and x86_64 implementations. |
| 264 | | |
| 265 | | **Duration:** 6 Months |
| 266 | | |
| 267 | | **Payment:** 30,000 EUR |
| 268 | | |
| 269 | | **Developer:** Ramiro Polla <ramiro.polla@gmail.com> |
| 270 | | |
| 271 | | I live in Belgium, and have been a contributor to FFmpeg since 2006. I am very familiar with libswscale, both the legacy codebase and the recent modernization project, which I have followed closely. I have been working on AArch64 NEON SIMD optimizations for the past couple of years. Besides contributing to open-source multimedia projects, I have also worked for five years at Hex-Rays, helping develop the IDA Pro reverse engineering software. |
| 272 | | |
| 273 | | ==== Milestones |
| 274 | | |
| 275 | | ===== '''asmjit''' validation backend |
| 276 | | |
| 277 | | * **Description**: A JIT validation backend will be made using the '''asmjit''' library. This will allow us to validate the generated assembly code and obtain performance metrics. |
| 278 | | |
| 279 | | * **Deliverables**: An initial '''asmjit''' JIT validation backend in a separate git repository. An RFC with a performance report posted to the FFmpeg-devel mailing list. |
| 280 | | |
| 281 | | * **Compensation:** 6,000 EUR |
| 282 | | |
| 283 | | ===== Assembly generation abstraction layer |
| 284 | | |
| 285 | | * **Description**: The '''asmjit''' validation backend will be split in two parts: an abstract assembly generation layer (with no '''asmjit''' dependency), and the '''asmjit''' code generator based on the abstraction layer. This assembly generation layer should include a register allocator. It will then be possible to add other assembly output generators. |
| 286 | | |
| 287 | | * **Deliverables**: An update on the RFC on the FFmpeg-devel mailing list with the code split. |
| 288 | | |
| 289 | | * **Compensation:** 6,000 EUR |
| 290 | | |
| 291 | | ===== Static non-JIT backend |
| 292 | | |
| 293 | | * **Description**: Using the assembly abstraction layer and the AArch64 NEON SIMD code from the asmjit validation backend, it will be possible to generate static assembly files that match the CPS code of the reference C and x86_64 implementations, even though they come with a certain overhead compared to the JIT code. A tool (program or script) will be created to generate the static assembly files. This tool will be integrated into the FFmpeg codebase, and the static assembly files will be generated either at build-time or integrated into the FFmpeg codebase as well (to be determined). |
| 294 | | |
| 295 | | * **Deliverables**: Series merged to git master, effectively providing an initial AArch64 NEON backend for the new libswscale architecture. |
| 296 | | |
| 297 | | * **Compensation:** 6,000 EUR |
| 298 | | |
| 299 | | ===== Evaluate other JIT implementations |
| 300 | | |
| 301 | | * **Description**: The '''asmjit''' validation backend, after having been used to bootstrap the static non-JIT backend, will be revisited. Other JIT generation backends will be evaluated, with a focus on either finding or creating a framework that can be cleanly integrated with FFmpeg's C codebase. This dependency should be lightweight and easy to integrate into FFmpeg itself, guaranteeing the maintainability of the new codebase. |
| 302 | | |
| 303 | | * **Deliverables**: An RFC with a report on the findings posted to the FFmpeg-devel mailing list. The community will help select the best implementation. |
| 304 | | |
| 305 | | * **Compensation:** 6,000 EUR |
| 306 | | |
| 307 | | ===== Final JIT backend |
| 308 | | |
| 309 | | * **Description**: The final AArch64 NEON JIT backend will be integrated into the FFmpeg codebase, based on the implementation selected by the community. Support will be added incrementally for each platform (Linux, macOS, Windows), only after they are each properly tested an validated. This JIT implementation will considerably reduce the overhead compared to the non-JIT code, ensuring that there are no performance regressions between the legacy code paths and the new architecture for AArch64 platforms. |
| 310 | | |
| 311 | | * **Deliverables**: A pull request with the final AArch64 NEON JIT backend, is either merged or has received at least one merge approval. |
| 312 | | |
| 313 | | * **Compensation:** 6,000 EUR |
| 314 | | |
| 315 | | === Fixes of fuzzer found bugs === |
| 316 | | |
| 317 | | '''One Line Summary:''' Fixing issues found by fuzzers |
| 318 | | |
| 319 | | '''Description:''' FFmpeg is tested by more and more automated systems, fuzzers, AI and soon a bug bounty programs. This task is to fund some of the efforts to triage and fix the stream of incoming issues |
| 320 | | |
| 321 | | '''Expected results:''' Triaging and fixing most newly incoming issues |
| 322 | | |
| 323 | | '''Duration:''' 9 Months |
| 324 | | |
| 325 | | '''Payment:''' 60000€ |
| 326 | | |
| 327 | | '''Developer:''' Michael Niedermayer (michael-stf@niedermayer.cc) |
| 328 | | |
| 329 | | ==== Developer background and contact information |
| 330 | | |
