Changes between Version 29 and Version 30 of SponsoringPrograms/STF/2025


Ignore:
Timestamp:
Sep 9, 2025, 2:15:33 AM (13 months ago)
Author:
Ramiro Polla
Comment:

Update "libswscale AArch64 NEON backend" proposal

Legend:

Unmodified
Added
Removed
Modified
  • SponsoringPrograms/STF/2025

    v29 v30  
    215215* **Compensation**: 10,000 EUR
    216216
    217 === swscale AArch64 NEON backend ===
    218 
    219 **One line summary:** This project will implement an AArch64 NEON backend for the new swscale framework.
    220 
    221 **Description:** The recent swscale rewrite has transformed it into a flexible and reliable framework to handle scaling, with the logic being decoupled from the DSP code. This simplifies writing backends with all the benefits and deterministic behaviour of the reference C implementation. This project will extend the new swscale framework by implementing an AArch64 NEON backend, ensuring that ARM-based platforms (including mobile devices, servers, and single-board computers) benefit from the same deterministic and optimized pipeline as x86_64. This backend will be used to generate optimized NEON SIMD assembly code at runtime for platforms where just-in-time code execution (JIT) is supported, and it will also be used to generate static (non-JIT) assembly files that match the Continuation-Passing Style (CPS) code of the reference C and x86_64 implementations.
    222 
    223 **Expected results:** swscale AArch64 NEON backend, both JIT and non-JIT, with feature parity to the reference C and x86_64 implementations.
     217=== libswscale AArch64 NEON backend ===
     218
     219**One line summary:** This project will implement an AArch64 NEON backend for the new libswscale architecture.
     220
     221**Description:** The recent libswscale rewrite is transforming the library into a flexible and reliable architecture to handle scaling, with the logic being decoupled from the DSP code. This simplifies writing backends with all the benefits and deterministic behaviour of the reference C implementation. In order to achieve the goal of deprecating and phasing out the "legacy" code paths from FFmpeg, the new architecture must not introduce performance regressions for platforms with widespread use, such as x86_64 and AArch64, for Linux, macOS, and Windows. This project will extend the new libswscale architecture by implementing an AArch64 NEON backend, ensuring that ARM-based platforms (including mobile devices, servers, and single-board computers) benefit from the same deterministic and optimized pipeline as x86_64. This backend will be used to generate highly optimized NEON SIMD assembly code at runtime for platforms where just-in-time code execution (JIT) is supported, and it will also be used to generate static (non-JIT) assembly files that match the Continuation-Passing Style (CPS) code of the reference C and x86_64 implementations for platforms where JIT is not supported.
     222
     223**Expected results:** libswscale AArch64 NEON backend, both JIT and non-JIT, with feature parity to the reference C and x86_64 implementations.
    224224
    225225**Duration:** 6 Months
    … …  
    229229**Developer:** Ramiro Polla <ramiro.polla@gmail.com>
    230230
     231I live in Belgium, and have been a contributor to FFmpeg since 2006. I am very familiar with libswscale, both the legacy codebase and the recent modernization project, which I have followed closely. I have been working on AArch64 NEON SIMD optimizations for the past couple of years. Besides contributing to open-source multimedia projects, I have also worked for five years at Hex-Rays, helping develop the IDA Pro reverse engineering software.
     232
    231233==== Milestones
    232234
    … …  
    237239* **Deliverables**: An initial '''asmjit''' JIT validation backend in a separate git repository. An RFC with a performance report posted to the FFmpeg-devel mailing list.
    238240
     241* **Compensation:** 6,000 EUR
     242
    239243===== Assembly generation abstraction layer
    240244
    … …  
    243247* **Deliverables**: An update on the RFC on the FFmpeg-devel mailing list with the code split.
    244248
     249* **Compensation:** 6,000 EUR
     250
    245251===== Static non-JIT backend
    246252
    247 * **Description**: Using the assembly abstraction layer and the AArch64 NEON SIMD code from the asmjit validation backend, it will be possible to generate static assembly files that match the Continuation-Passing Style code of the reference C and x86_64 implementations. A tool (program or script) will be created to generate the static assembly files. This tool will be integrated into the FFmpeg codebase, and the static assembly files will be generated either at build-time or integrated into the FFmpeg codebase as well (to be determined).
    248 
    249 * **Deliverables**: Series merged to git master, effectively providing an initial AArch64 NEON backend for the new swscale framework.
     253* **Description**: Using the assembly abstraction layer and the AArch64 NEON SIMD code from the asmjit validation backend, it will be possible to generate static assembly files that match the CPS code of the reference C and x86_64 implementations, even though they come with a certain overhead compared to the JIT code. A tool (program or script) will be created to generate the static assembly files. This tool will be integrated into the FFmpeg codebase, and the static assembly files will be generated either at build-time or integrated into the FFmpeg codebase as well (to be determined).
     254
     255* **Deliverables**: Series merged to git master, effectively providing an initial AArch64 NEON backend for the new libswscale architecture.
     256
     257* **Compensation:** 6,000 EUR
    250258
    251259===== Evaluate other JIT implementations
    … …  
    255263* **Deliverables**: An RFC with a report on the findings posted to the FFmpeg-devel mailing list. The community will help select the best implementation.
    256264
     265* **Compensation:** 6,000 EUR
     266
    257267===== Final JIT backend
    258268
    259 * **Description**: The final AArch64 NEON JIT backend will be integrated into the FFmpeg codebase, based on the implementation selected by the community. Support will be added incrementally for each platform, only after they are each properly tested an validated.
    260 
    261 * **Deliverables**: A pull request to be reviewed by the FFmpeg community.
     269* **Description**: The final AArch64 NEON JIT backend will be integrated into the FFmpeg codebase, based on the implementation selected by the community. Support will be added incrementally for each platform (Linux, macOS, Windows), only after they are each properly tested an validated. This JIT implementation will considerably reduce the overhead compared to the non-JIT code, ensuring that there are no performance regressions between the legacy code paths and the new architecture for AArch64 platforms.
     270
     271* **Deliverables**: A pull request with the final AArch64 NEON JIT backend, is either merged or has received at least one merge approval.
     272
     273* **Compensation:** 6,000 EUR
    262274
    263275=== Fixes of fuzzer found bugs ===