Context Navigation

2024

Timestamp:: Feb 12, 2024, 7:34:56 PM (2 years ago)
Author:: Pierre-Anthony Lemieux
Comment:: Update proposal with SOWs from Michael Niedermayer and Niklas Haas

Legend:

: Unmodified
: Added
: Removed
: Modified

SponsoringPrograms/STF/2024

-              v30
+              v31
 **Purpose:** Prepare a project description for review board.
 ### **1) Describe the technology this effort will support.**
+== **1) Describe the technology this effort will support.**
 The proposed effort aims to support FFmpeg, a leading multimedia framework recognized for its versatile capabilities.
 …
 FFmpeg serves as a comprehensive solution for decoding, encoding, transcoding, multiplexing, demultiplexing, streaming, filtering, and playback of a wide array of multimedia formats. It stands out by seamlessly handling formats from ancient, obscure standards to cutting-edge ones, making it a cornerstone in the multimedia processing landscape.
 ### **2) Where is this technology being used? Why is it relevant and critical? How does this technology serve the public interest?**
+== **2) Where is this technology being used? Why is it relevant and critical? How does this technology serve the public interest?**
 FFmpeg is a ubiquitous technology, deeply embedded in the fabric of modern multimedia processing and touching the lives of millions every day. FFmpeg is an integral component in a vast array of end-user devices, including desktops, laptops, mobile phones, smartwatches, and smart TVs. FFmpeg also powers major video distribution and streaming platforms like YouTube, Vimeo, AWS, and Microsoft Azure, shaping the landscape of online content consumption.
 …
 The relevance and criticality of FFmpeg lies in its role as the backbone of multimedia processing. It serves the public interest by facilitating the seamless functioning of everyday technologies, from video playback in web browsers like Firefox and Chromium, to applications like Kodi, MPlayer, and OBS Studio. Furthermore, FFmpeg plays a crucial role in public sector use cases, website interactions, and even in the transmission of multimedia content within emails. In essence, FFmpeg's impact is far-reaching, touching both the private and public spheres. FFmpeg's reliability is paramount for the secure and efficient functioning of digital multimedia.
 ### **3) What are the problems the proposed work is trying to address? What activities do you propose and how do they address the problems described? How do those activities fit into [STF’s mission](https://www.sovereigntechfund.de/mission)?**
+== **3) What are the problems the proposed work is trying to address? What activities do you propose and how do they address the problems described? How do those activities fit into [STF’s mission](https://www.sovereigntechfund.de/mission)?**
 The proposed work addresses critical challenges in maintaining FFmpeg's sustainability, security, and innovation. Activities include:
 …
 These activities align with STF’s mission by supporting the maintenance and security of open source components and the strengthening the Open Source ecosystem.
 ### **4) Who is going to be performing the work and how are they qualified to do so?**
+== **4) Who is going to be performing the work and how are they qualified to do so?**
 The work will be performed by key FFmpeg project contributors and other experienced developers. Their qualifications have been demonstrated through years of active involvement in FFmpeg development, participating in diverse tasks including code contributions, code review, user support, and administration.
 ### **5) How is the technology maintained or governed? Is there a community behind the technology, and do they approve of the work?**
+== **5) How is the technology maintained or governed? Is there a community behind the technology, and do they approve of the work?**
 FFmpeg's maintenance and governance in based on a flat hierarchy, with decisions being made transparently through written communications in mailing lists.
 …
 The proposed work aligns with the community's focus on quality, security, and openness and will/has been approved following the community's operating governance processes.
+### **6) Describe the Milestones Using the Template below**
+**Milestone 1:** Ecosystem and Infrastructure Support
+**Cost:**
+**Duration:** Throughout the Project (12 months)
+In this initial milestone, the focus is on fortifying FFmpeg's security posture and ensuring the robustness of its infrastructure. This milestone also includes enhancements in other areas like user support on this TRAC system and the mailing lists.
+This milestone's activities include:
+- **Security Fixes and Hardening (€??.???):** Fix the majority of coverity issues (674 issues), fix coverage testing of existing oss-fuzz in some form (possibly on separate infrastructure)
+- **Administration of FFmpeg Infrastructure (€??.???):** Enhancing the overall administration experience and the reliability of FFmpeg's infrastructure.
+- **User Support via TRAC and Mailing List (€??.???):** Streamlining user support mechanisms through TRAC and the user mailing list.
+- **Reviewing Code Submissions (€??.???):** Ensuring the quality and security of code submissions through a robust review process.
+**Milestone 2:** Enhancing Existing Implementations (€??.???)
+This milestone is dedicated to advancing existing codecs, formats, and filters within FFmpeg. Activities include:
+- **Improving ABCD (€5,000):** Enhancing the functionality and efficiency of ABCD. (X Months)
+- **Improving EFGH (AAAA) Muxer/Demuxer (€7,000):** Addressing inefficiencies and improving the handling of EFGH formats. (Y Months)
+### **7) Please fill out the following Supplementary Information**
+- **Preferred Mode of Invoicing:**
+- **Contracting Entity (Is it a fiscal host?):** SPI
+- **Total Budget Requested:**
+- **Requested Fully Loaded Hourly Rate (Total cost divided by number of person hours):**
+- **Project Actual Duration:** 12 Months
+- **Earliest possible starting date:**
+== Template 1 ==
+'''Description:''' Improve the foo bar
+'''Expected results:''' a foo bar that has fully thread safe logging
+'''Duration:''' 350 hours
+'''Payment:''' 28,000€
+'''Developer:''' Donald Duck
+== Template 2 ==
+'''Description:''' Work on improving the bar foo
+'''Expected results:''' Spend 6 month fulltime work on improving bar foo, fixing bugs and regression people report, refactor the code to make it more maintainable
+'''Duration:''' 6 Months
+'''Payment:''' 35,000€
+'''Developer:''' NA
+== **6) Describe the Milestones Using the Template below**
+=== Classify and fix outstanding issues identified by Coverity and improve related code.
+==== Overview
+Coverity is a static code analysis system that is used to analyze FFmpeg code
+to find bugs with an emphasis on quality and security issues. There are
+currently 673 outstanding issues identified by Coverity
+(https://scan.coverity.com/projects/ffmpeg?tab=overview). Some of these
+issues are false positives while others could open the door to security
+vulnerabilities.
+The objective of this work is to identify the Coverity issues that are
+not false positives, and fix 90% of them, split into three tranches of 30%.
+Any issue that require modules rewrite or redesign are outside the scope of
+this SoW. Any related issues will be fixed / cleaned up if practical. An
+example of this is the use of literal 4 instead of a named identifier for the
+number of planes in a false positive "out of array access" issue. The issue
+was closed as false positive yet patches to make the code more robust and
+easier to understand where submitted none the less.
+**Developer**: Michael Niedermayer <michael-ffwork@niedermayer.cc>
+**Duration**: 12 months
+==== Milestones
+===== Milestone C
+* **Description**: Review all outstanding Coverity issues and, for each one, determine whether it is a false positive.
+* **Deliverables**: List of both false positive and potentially real issues posted to the FFMPEG dev mailing list.
+===== Milestone S1
+* **Description**. Fix the first 30% tranche of outstanding real issues.
+* **Deliverables**. Patches submitted for review to the FFMPEG dev mailing list.
+===== Milestone M1
+* **Description**. Commit to the FFMPEG master branch 90% of the changes from S1.
+* **Deliverables**. Link to the commits containing the changes.
+===== Milestone S2
+* **Description**. Fix the second 30% tranche of outstanding real issues.
+* **Deliverables**. Patches submitted for review to the FFMPEG dev mailing list.
+===== Milestone M2
+* **Description**. Commit to the FFMPEG master branch 90% of the changes from S2.
+* **Deliverables**. Link to the commits containing the changes.
+===== Milestone S2
+* **Description**. Fix the third 30% tranche of outstanding real issues.
+* **Deliverables**. Patches submitted for review to the FFMPEG dev mailing list.
+===== Milestone M2
+* **Description**. Commit to the FFMPEG master branch 90% of the changes from S2.
+* **Deliverables**. Link to the commits containing the changes.
+=== Maintain the FFV1 Codec
+==== Overview
+FFv1 has become one of the entirely open and free de facto standard lossless
+codecs for video archival. There has however been insufficient resources to
+maintain it and there are, as a result, several outstanding issues and pull
+requests to address:
+https://github.com/FFmpeg/FFV1/issues
+https://github.com/FFmpeg/FFV1/pulls
+The objective of the work is
+**Developer**: Michael Niedermayer <michael-ffwork@niedermayer.cc>
+**Duration**: 12 months
+==== Milestones
+===== Fuzzing
+* **Description**: Add encoder and decoder fuzzing to find new security issues.
+* **Deliverables**: Scripts and input files necessary for fuzzing
+===== Bug fix
+* **Description**. Fix 90% of outstanding bugs and merge 90% of outstanding pull requests intended to fix outstanding bugs.
+* **Deliverables**. Link to commits on the FFMPEG master branch containing the changes.
+===== Add support for floating point samples
+* **Description**. The objective is to add support for floating point samples
+to FFv1. This would allow lossless storage of a wider range of content in
+archives using FFv1.
+* **Deliverables**. List of git commits on the FFMPEG master branch adding
+broadening the support of formats in the specification, encoder and decoder
+=== Improving and modernizing libswscale
+==== Overview
+libswscale is an old and integral component of FFmpeg which is relied on,
+in some form, by almost every user. Despite this, it is one of the hardest
+parts of FFmpeg to extend and understand, owing to its historical growth
+in complexity. The aim of this project will be to alleviate some of the
+issues and start work on bringing libswscale into a cleaned up, more
+maintainable state.
+**Developer**: Niklas Haas <ffmpeg@haasn.xyz>
+**Duration**: 12 months
+==== Milestones
+===== Design new high-level API
+* **Description**: Develop and design a new high-level API
+(libswscale/avscale.h), which is capable of abstracting away the internal
+messy, stateful details of libswscale. This API *should* ideally also be
+written in such a way that we can internally use zscale or libplacebo instead
+of libswscale, possibly even at the same time (e.g. if incompatible, complex,
+multi-step transformations are requested), be almost entirely stateless
+(backed by some sort of "state cache" object?), and allow for implementing
+things like slice or frame threading at a higher level (thus freeing
+libswscale from the burden of managing cascaded contexts internally).
+* **Deliverables**: RFC of complete API draft posted to mailing list, with at
+least general consensus that this API represents a step in the right
+direction.
+===== Implement backends for new API
+* **Description**. The previously designed API should be fully implemented,
+hooked up to at least libswscale, in particular, in a way that solves the
+existing issues of libswscale's statefulness (e.g. errors after reconfiguring
+to different bit depths or color spaces). If time allows, also hook it up to
+libplacebo and/or zscale as a proof of concept, possibly just to provide an
+initial backing for operations which libswscale does not yet support (e.g.
+colorspace conversions, GPU frame support).
+* **Deliverables**. Series merged to git master introducing new API and backing
+implementation, vf_scale successfully ported to avscale.h.
+===== Extend and fix regression tests
+* **Description**. Fix existing known issues with libswscale's self-tests,
+and extend them by adding coverage for more paths. In particular, add
+checkasm support for existing assembly primitives. Fix any issues that are
+uncovered by the additional tests.
+* **Deliverables**. Patch series merged to git master, closing existing
+issues (e.g.  #10824), adding testing for at least half of assembly routines,
+and fixing whatever is needed to ensure that the assembly passes checkasm.
+===== Remove legacy code paths
+* **Description**. In the interest of making subsequent refactoring work more
+achievable, libswscale should first be slimmed down to the essentials of
+what's needed for reasonably modern platforms. Older code, assembly routines
+for rarely used platforms, legacy speed hacks like runtime-generated MMX code
+etc. should be identified, analyzed to figure out what actual benefit they
+provide, and proposed for removal.
+* **Deliverables**. A list of proposed removals, plus justification (e.g. in
+the form of benchmarks or platform statistics) is posted to the mailing list.
+All proposals are either accepted by the community (and merged), or rejected
+(and discarded).