| | 215 | == MPEG-4 ALS encoder == |
| | 216 | ATunes.png |
| | 217 | |
| | 218 | |
| | 219 | '''Description:''' A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on github. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the reference codec and specifications. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project. |
| | 220 | |
| | 221 | '''Expected results:''' |
| | 222 | |
| | 223 | - Update the existing encoder to fit into the current codebase. |
| | 224 | - Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE. |
| | 225 | - Ensure the FFmpeg decoder processes all generated files without warnings. |
| | 226 | - Enhance the rudimentary feature set of the encoder. |
| | 227 | |
| | 228 | '''Prerequisites:''' C coding skills, familiarity with git/source code control systems. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial. |
| | 229 | |
| | 230 | '''Qualification Task''': Add floating point support to MPEG-4 ALS decoder |
| | 231 | |
| | 232 | '''Mentor:''' TBA, possibly Paul B Mahol (durandal_1707 on IRC) |
| | 233 | |
| | 234 | '''Backup Mentor:''' TBA, possibly Stefano Sabatini (saste on IRC) |
| | 235 | |
| | 236 | |
| | 237 | == Hardware Acceleration (hwaccel) API v2 == |
| | 238 | Image:Hardware.jpg |
| | 239 | |
| | 240 | '''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwacel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification. |
| | 241 | |
| | 242 | The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace. |
| | 243 | |
| | 244 | '''Expected results:''' |
| | 245 | |
| | 246 | - FFmpeg core library (libavcodec): |
| | 247 | - Core API extensions and improvements |
| | 248 | - Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications |
| | 249 | - Add new tokens describing hardware accelerators |
| | 250 | - Add new flags exposing HW capabilities like download/upload |
| | 251 | - Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers |
| | 252 | - Port hwaccels to v2 infrastructure |
| | 253 | - Port VA-API decoders to v2 infrastructure |
| | 254 | - Validate that VA-API decoders still work with existing applications supporting hwaccel v1 |
| | 255 | - Provide download capability through vaGetImage() |
| | 256 | - Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer |
| | 257 | - Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications |
| | 258 | |
| | 259 | - FFmpeg applications: |
| | 260 | - Integrate hardware acceleration into ffplay |
| | 261 | - Create a video-output (VO) infrastructure to ffplay |
| | 262 | - Port the SDL renderer to the new VO infrastructure |
| | 263 | - Add support for VA-API: VA renderer through vaPutSurface(), add -hwaccel option to select "vaapi" renderer |
| | 264 | - Add support for VDPAU (optional): VDPAU renderer through VdpPresentationQueueDisplay() |
| | 265 | - Integrate hardware acceleration into ffmpeg |
| | 266 | - Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use |
| | 267 | |
| | 268 | '''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API. |
| | 269 | |
| | 270 | '''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc. |
| | 271 | |
| | 272 | '''Mentor:''' TBA, possibly Gwenole Beauchesne (__gb__ on IRC) |
| | 273 | |
| | 274 | '''Backup Mentor:''' TBA, possibly Hendrik Leppkes (nevcairiel on IRC) |
| | 275 | |
| | 276 | |
| | 277 | == Hardware Accelerated Video Encoding with VA-API == |
| | 278 | |
| | 279 | '''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new hwaccel hooks are needed and two operational modes are possible: either (i) driver or hardware pack headers themselves, or (ii) lattitude is left to perform this task at the FFmpeg library level. |
| | 280 | |
| | 281 | '''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested. |
| | 282 | |
| | 283 | - MPEG-2 encoding: |
| | 284 | - Add basic encoding with I/P frames (handle the -g option) |
| | 285 | - Add support for B frames (handle the -bf option) |
| | 286 | - Add support for constant bitrate (CBR, i.e. maxrate == bitrate and bufsize set) |
| | 287 | - (Optionally) add support for interlaced contents |
| | 288 | - H.264 encoding: |
| | 289 | - Add basic encoding with I/P frames (handle the -g option) |
| | 290 | - Add support for B frames (handle the -bf option) |
| | 291 | - Add support for constant bitrate (CBR, i.e. maxrate == bitrate and bufsize set) |
| | 292 | - Add support for constant QP (CQP, i.e. handle the -cqp option) |
| | 293 | - Add support for more than one reference frame, while providing/using API to query the hardware capabilities |
| | 294 | - Work on HRD conformance. May require to write an independent tool to assess that |
| | 295 | - (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator. |
| | 296 | - FFmpeg applications: |
| | 297 | - Define common hwaccel interface for encoding |
| | 298 | - Add initial support for hardware accelerated encoding to the ffmpeg application |
| | 299 | |
| | 300 | '''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API for encoding. |
| | 301 | |
| | 302 | '''Qualification Task''': Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc. |
| | 303 | |
| | 304 | '''Mentor:''' TBA, possibly Gwenole Beauchesne (__gb__ on IRC) |
| | 305 | |
| | 306 | '''Backup Mentor:''' TBA, possibly Tushar Gohad |
| | 307 | |
| | 308 | |
| | 309 | == AAC Improvements == |
| | 310 | |
| | 311 | '''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work. |
| | 312 | |
| | 313 | - AAC LD decoder |
| | 314 | - AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples |
| | 315 | - AAC SSR decoder |
| | 316 | - AAC 960/120 MDCT window |
| | 317 | - AAC multi-channel encoding |
| | 318 | |
| | 319 | '''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks. |
| | 320 | |
| | 321 | '''Prerequisites:''' C coding skills, familiarity with git/source code control systems, knowledge about transform based audio coding would be useful. |
| | 322 | |
| | 323 | '''Mentor:''' TBA, possibly Baptiste Coudurier (bcoudurier on IRC) |
| | 324 | |
| | 325 | '''Backup Mentor:''' TBA, possibly Stefano Sabatini (saste on IRC) |
| | 326 | |
| | 327 | |
| | 328 | == DTS / DCA Improvements == |
| | 329 | |
| | 330 | '''Description:''' FFmpeg contains a DTS decoder. |
| | 331 | |
| | 332 | - DTS-HD decoder improvements: A possible qualification task is to implement ticket #1920 |
| | 333 | - Add support for X96 extension (96khz) |
| | 334 | - Add support for XLL extension (lossless) |
| | 335 | - Add support for pure DTS-HD streams that do not contain a DTS core |
| | 336 | - Add support for multiple assets |
| | 337 | - Add support for LBR extension |
| | 338 | |
| | 339 | '''Prerequisites:''' C coding skills, familiarity with git/source code control systems. |
| | 340 | |
| | 341 | '''Mentor:''' TBA, possibly Benjamin Larsson (merbanan on IRC) |
| | 342 | |
| | 343 | '''Backup Mentor:''' TBA, possibly Stefano Sabatini (saste on IRC) |
| | 344 | |
| | 345 | |
