Changes between Version 23 and Version 24 of SponsoringPrograms/STF/2025


Ignore:
Timestamp:
Aug 25, 2025, 1:34:35 PM (10 months ago)
Author:
Ramiro Polla
Comment:

Improve swscale AArch64 NEON backend proposal

Legend:

Unmodified
Added
Removed
Modified

  • SponsoringPrograms/STF/2025

    v23 v24  
    177177**One line summary:** This project will implement an AArch64 NEON backend for the new swscale framework.
    178178
    179 **Description:** The recent swscale rewrite has transformed it into a flexible and reliable framework to handle scaling, with the logic being decoupled from the DSP code. This simplifies writing backends with all the benefits and deterministic behaviour of the reference C implementation. This project will add an AArch64 NEON backend that generates optimized NEON SIMD assembly. This backend will be used to generate code at runtime for platforms where just-in-time code execution (JIT) is supported, and it will also be used to generate static assembly files that match the Continuation-Passing Style (CPS) code of the reference C and x86 implementations.
    180 
    181 **Expected results:** swscale AArch64 NEON backend, both JIT and non-JIT, with feature parity to the reference C and x86 implementations.
     179**Description:** The recent swscale rewrite has transformed it into a flexible and reliable framework to handle scaling, with the logic being decoupled from the DSP code. This simplifies writing backends with all the benefits and deterministic behaviour of the reference C implementation. This project will extend the new swscale framework by implementing an AArch64 NEON backend, ensuring that ARM-based platforms (including mobile devices, servers, and single-board computers) benefit from the same deterministic and optimized pipeline as x86_64. This backend will be used to generate optimized NEON SIMD assembly code at runtime for platforms where just-in-time code execution (JIT) is supported, and it will also be used to generate static (non-JIT) assembly files that match the Continuation-Passing Style (CPS) code of the reference C and x86_64 implementations.
     180
     181**Expected results:** swscale AArch64 NEON backend, both JIT and non-JIT, with feature parity to the reference C and x86_64 implementations.
    182182
    183183**Duration:** 6 Months
    … …  
    189189==== Milestones
    190190
    191 ===== '''asmjit''' backend
    192 
    193 * **Description**: A JIT proof-of-concept will be made using the '''asmjit''' library. This will allow us to validate the generated assembly code and obtain performance metrics.
    194 
    195 * **Deliverables**: An initial '''asmjit''' JIT proof-of-concept in a separate git repository. An RFC with a performance report posted to the FFmpeg-devel mailing list.
     191===== '''asmjit''' validation backend
     192
     193* **Description**: A JIT validation backend will be made using the '''asmjit''' library. This will allow us to validate the generated assembly code and obtain performance metrics.
     194
     195* **Deliverables**: An initial '''asmjit''' JIT validation backend in a separate git repository. An RFC with a performance report posted to the FFmpeg-devel mailing list.
    196196
    197197===== Assembly generation abstraction layer
    198198
    199 * **Description**: The '''asmjit''' proof-of-concept will be split in two parts: an abstract assembly generation layer (with no '''asmjit''' dependency), and the '''asmjit''' code generator based on the abstract layer. This assembly generation layer should include a register allocator. It will then be possible to add other assembly output generators.
     199* **Description**: The '''asmjit''' validation backend will be split in two parts: an abstract assembly generation layer (with no '''asmjit''' dependency), and the '''asmjit''' code generator based on the abstraction layer. This assembly generation layer should include a register allocator. It will then be possible to add other assembly output generators.
    200200
    201201* **Deliverables**: An update on the RFC on the FFmpeg-devel mailing list with the code split.
    … …  
    203203===== Static non-JIT backend
    204204
    205 * **Description**: Using the assembly abstraction layer, it will be possible to generate static assembly files that match the Continuation-Passing Style (CPS) code of the reference C and x86 implementations. A tool (program or script) will be created to generate the static assembly files. This tool will be integrated into the FFmpeg codebase, and the static assembly files will be generated either at build-time or integrated into the FFmpeg codebase as well (to be determined).
    206 
    207 * **Deliverables**: Series merged to git master, effectively providing an initial AArch64 NEON backend.
    208 
    209 ===== Explore other JIT implementations
    210 
    211 * **Description**: '''asmjit''' will not necessarily be the definitive JIT implementation. It will initially serve merely as a proof-of-concept, and as a springboard to the static non-JIT backend. At this point, other JIT generation backends will be explored, with a focus on either finding or creating a framework that can be cleanly integrated with FFmpeg's C codebase. This dependency should be lightweight or easy to integrate into FFmpeg itself.
     205* **Description**: Using the assembly abstraction layer and the AArch64 NEON SIMD code from the asmjit validation backend, it will be possible to generate static assembly files that match the Continuation-Passing Style code of the reference C and x86_64 implementations. A tool (program or script) will be created to generate the static assembly files. This tool will be integrated into the FFmpeg codebase, and the static assembly files will be generated either at build-time or integrated into the FFmpeg codebase as well (to be determined).
     206
     207* **Deliverables**: Series merged to git master, effectively providing an initial AArch64 NEON backend for the new swscale framework.
     208
     209===== Evaluate other JIT implementations
     210
     211* **Description**: The '''asmjit''' validation backend, after having been used to bootstrap the static non-JIT backend, will be revisited. Other JIT generation backends will be evaluated, with a focus on either finding or creating a framework that can be cleanly integrated with FFmpeg's C codebase. This dependency should be lightweight and easy to integrate into FFmpeg itself, guaranteeing the maintainability of the new codebase.
    212212
    213213* **Deliverables**: An RFC with a report on the findings posted to the FFmpeg-devel mailing list. The community will help select the best implementation.