| 68 | | '''Description:''' Add support for Animated PNGs. The little bouncing ball animation shown to the right is such an APNG file. |
| 69 | | |
| 70 | | Specification: https://wiki.mozilla.org/APNG_Specification |
| 71 | | |
| 72 | | '''Expected results:''' |
| 73 | | |
| 74 | | |
| 75 | | |
| 76 | | - PNG images are not misdetected as APNG animations |
| 77 | | - APNG animations are not misdetected as PNG images |
| 78 | | |
| 79 | | |
| 80 | | |
| 81 | | - test code under valgrind so no invalid reads/writes happen |
| 82 | | |
| 83 | | |
| 84 | | |
| 85 | | - implement parsing of all APNG chunks (acTL, fcTL, fdAT) |
| 86 | | |
| 87 | | |
| 88 | | |
| 89 | | |
| 90 | | - test code under valgrind so no invalid reads/writes happen |
| 91 | | |
| 92 | | - APNG muxer & APNG encoder |
| 93 | | |
| 94 | | |
| 95 | | - add test for FATE |
| 96 | | |
| 97 | | '''Prerequisites:''' C coding skills, familiarity with git/source code control systems. |
| | 54 | '''Description:''' Add support for Animated PNGs. The little bouncing ball animation shown to the right is such an APNG file. |
| | 55 | |
| | 56 | '''Specification:''' https://wiki.mozilla.org/APNG_Specification |
| | 57 | |
| | 58 | '''Expected results:''' |
| | 59 | |
| | 60 | - APNG demuxer |
| | 61 | - implement robust probing: |
| | 62 | - PNG images are not misdetected as APNG animations |
| | 63 | - APNG animations are not misdetected as PNG images |
| | 64 | - splits stream into sensible packets (so they can be easily reused in APNG muxer) |
| | 65 | - survives fuzzing (zzuf) |
| | 66 | - add FATE coverage, coverage should be at least 70% |
| | 67 | - test code under valgrind so no invalid reads/writes happen |
| | 68 | |
| | 69 | - APNG decoder |
| | 70 | - use existing PNG decoder code (write decoder in same file) |
| | 71 | - implement parsing of all APNG chunks (acTL, fcTL, fdAT) |
| | 72 | - error handling |
| | 73 | - survives fuzzing (zzuf) |
| | 74 | - add test for FATE, coverage should be at least 75% |
| | 75 | - CRC checksum validation |
| | 76 | - test code under valgrind so no invalid reads/writes happen |
| | 77 | |
| | 78 | - APNG muxer && APNG encoder |
| | 79 | - use existing PNG encoder code (write encoder in same file) |
| | 80 | - write compliant files, make sure they play correctly in major web browsers that support APNG |
| | 81 | - add test for FATE |
| | 82 | |
| | 83 | '''Prerequisites:''' C coding skills, basic familiarity with git. |
| 116 | | '''Mentor:''' Michael Niedermayer (michaelni at #ffmpeg-devel IRC) |
| 117 | | |
| 118 | | '''Backup mentor:''' TBA |
| 119 | | |
| | 102 | '''Mentor:''' Michael Niedermayer (''michaelni'' in #ffmpeg-devel on Freenode IRC) |
| | 103 | |
| | 104 | '''Backup mentor:''' TBA |
| | 105 | |
| | 106 | == Subtitles == |
| | 107 | |
| | 108 | [[Image(subtitles_support.jpg, 300, right, nolink)]] |
| | 109 | |
| | 110 | '''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form. |
| | 111 | |
| | 112 | '''Expected results:''' |
| | 113 | - Add support for new subtitles formats. Example: a demuxer for .SUP files, just like !VobSub but for Blu-Ray, or a !VobSub muxer. |
| | 114 | - Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT. |
| | 115 | - Update the API to get rid of the clumsy internal text representation of styles |
| | 116 | - Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay. |
| | 117 | - BONUS: if everything goes well, the student will be allowed to add basic support for teletext |
| | 118 | |
| | 119 | '''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required. |
| | 120 | |
| | 121 | '''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood. |
| | 122 | |
| | 123 | '''Mentor:''' Clément Bœsch (''ubitux'' in #ffmpeg-devel on Freenode IRC) |
| | 124 | |
| | 125 | '''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC) |
| | 126 | |
| | 127 | |
| | 128 | == Postproc optimizations == |
| | 129 | |
| | 130 | [[Image(PostProc.jpg, 240, right, nolink)]] |
| | 131 | |
| | 132 | |
| | 133 | '''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+). |
| | 134 | |
| | 135 | '''Expected results:''' |
| | 136 | - Convert all gcc inline asm in libpostproc to YASM. |
| | 137 | - Restructure the code so that it works with block sizes compatible with modern SIMD. |
| | 138 | - Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc. |
| | 139 | |
| | 140 | '''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git. |
| | 141 | |
| | 142 | '''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2. |
| | 143 | |
| | 144 | '''Mentor:''' Michael Niedermayer (''michaelni'' in #ffmpeg-devel on Freenode IRC) |
| | 145 | |
| | 146 | '''Backup Mentor:''' TBA |
| | 147 | |
| | 148 | |
| | 149 | == Bayer RGB colorspaces == |
| | 150 | |
| | 151 | [[Image(bayer_rgb_colorspaces.png, right, nolink)]] |
| | 152 | |
| | 153 | '''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing. |
| | 154 | |
| | 155 | '''Expected results:''' |
| | 156 | - Rebase existing patches |
| | 157 | - Implement high quality bayer transformations in libswscale (plain C) |
| | 158 | - Add bayer formats to the libavutil pixfmt enumeration routines |
| | 159 | - SIMD optimizations of the libswscale transformations |
| | 160 | - Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing) |
| | 161 | |
| | 162 | Optional goodies: |
| | 163 | - Extend TIFF decoder to support DNG-Bayer format |
| | 164 | - Support a popular proprietary camera format (many to choose from; see dcraw project) |
| | 165 | |
| | 166 | '''Prerequisites:''' C coding skills, basic familiarity with git. |
| | 167 | |
| | 168 | '''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale |
| | 169 | |
| | 170 | '''Mentor:''' Peter Ross (''pross-au'' in #ffmpeg-devel on Freenode IRC) |
| | 171 | |
| | 172 | '''Backup Mentor:''' Michael Niedermayer (''michaelni'' in #ffmpeg-devel on Freenode IRC) |
| | 173 | |
| | 174 | |
| | 175 | == MPEG-4 Audio Lossless Coding (ALS) encoder == |
| | 176 | |
| | 177 | [[Image(showwaves_green.png, 240, right, nolink)]] |
| | 178 | |
| | 179 | '''Description:''' |
| | 180 | A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project. |
| | 181 | |
| | 182 | '''Expected results:''' |
| | 183 | |
| | 184 | - Update the existing encoder to fit into the current codebase. |
| | 185 | - Ensure conformance of the encoder by verifying using the reference codec and generate a test case for [http://ffmpeg.org/fate.html FATE]. |
| | 186 | - Ensure the FFmpeg decoder processes all generated files without warnings. |
| | 187 | - Enhance the rudimentary feature set of the encoder. |
| | 188 | |
| | 189 | '''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial. |
| | 190 | |
| | 191 | '''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder |
| | 192 | |
| | 193 | '''Mentor:''' Paul B Mahol (''durandal_1707'' in #ffmpeg-devel on Freenode IRC) |
| | 194 | |
| | 195 | '''Backup Mentor:''' TBA |
| | 196 | |
| | 197 | |
| | 198 | == Hardware Acceleration API Software / Tracing Implementation == |
| | 199 | |
| | 200 | [[Image(hwaccel.jpg, right, nolink)]] |
| | 201 | |
| | 202 | '''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps, and in part since it requires specific operating systems and hardware. |
| | 203 | |
| | 204 | The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and data to allow easy comparison and regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what [https://github.com/apitrace/apitrace apitrace] does for OpenGL. |
| | 205 | |
| | 206 | A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware. |
| | 207 | |
| | 208 | '''Prerequisites:''' C coding skills, basic familiarity with git |
| | 209 | |
| | 210 | '''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable |
| | 211 | |
| | 212 | '''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de) |
| | 213 | |
| | 214 | '''Backup Mentor:''' TBA |
| | 215 | |
| | 216 | == AAC Improvements == |
| | 217 | |
| | 218 | '''Description:''' FFmpeg contains an AAC encoder and decoder, and both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work. |
| | 219 | |
| | 220 | - AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples |
| | 221 | - AAC SSR decoder |
| | 222 | - AAC 960/120 MDCT window |
| | 223 | |
| | 224 | '''Qualification Task:''' See the [https://trac.ffmpeg.org/query?status=!closed&keywords=~aac FFmpeg bug tracker for AAC issues], fixing one of them or rebasing the existing incomplete BSAC decoder for current git head, or fixing one or more existing bugs are possible qualification tasks. |
| | 225 | |
| | 226 | '''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful. |
| | 227 | |
| | 228 | '''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC) |
| | 229 | |
| | 230 | '''Backup Mentor:''' TBA |
| | 231 | |
| | 232 | |
| | 233 | == DTS / DCA Improvements == |
| | 234 | |
| | 235 | '''Description:''' FFmpeg contains a DTS decoder, but its missing several features |
| | 236 | |
| | 237 | - DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920] |
| | 238 | - Add support for X96 extension (96khz) |
| | 239 | - Add support for XLL extension (lossless) |
| | 240 | - Add support for pure DTS-HD streams that do not contain a DTS core |
| | 241 | - Add support for multiple assets |
| | 242 | - Add support for LBR extension |
| | 243 | |
| | 244 | '''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. |
| | 245 | |
| | 246 | '''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC) |
| | 247 | |
| | 248 | '''Backup Mentor:''' TBA |
| | 249 | |
| | 250 | |
| | 251 | == MXF Demuxer Improvements == |
| | 252 | |
| | 253 | '''Description:''' The MXF demuxer needs a proper, compact way to map !EssenceContainer ULs to !WrappingKind. See [https://trac.ffmpeg.org/ticket/2776 ticket #2776] in our bug tracker, and [https://trac.ffmpeg.org/ticket/2776 ticket #1916] contains additional relevant information. |
| | 254 | |
| | 255 | Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary. |
| | 256 | |
| | 257 | '''Expected results:''' The sample in ticket [https://trac.ffmpeg.org/ticket/2776 ticket #2776] should demux correctly. Add a test case in [http://ffmpeg.org/fate.html FATE]. The solution should grow libavformat by no more than 32 KiB. |
| | 258 | |
| | 259 | '''Prerequisites:''' C coding skills, basic familiarity with git. |
| | 260 | |
| | 261 | '''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> !WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 Excel document]. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat. |
| | 262 | |
| | 263 | '''Mentor:''' Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC) |
| | 264 | |
| | 265 | '''Backup Mentor:''' TBA |
| | 266 | |
| | 267 | |
| | 268 | == Symmetric-key block ciphers == |
| | 269 | |
| | 270 | '''Description:''' |
| | 271 | FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia. |
| | 272 | |
| | 273 | '''Specification:''' |
| | 274 | - http://tools.ietf.org/search/rfc2144 |
| | 275 | - https://www.schneier.com/twofish.html |
| | 276 | - http://tools.ietf.org/html/rfc3713 |
| | 277 | |
| | 278 | '''Expected Results:''' |
| | 279 | - Implement CAST-128, Twofish and Camellia in C |
| | 280 | - Support for ECB and CBC modes |
| | 281 | - Ensure code works with all key sizes supported by an algorithm |
| | 282 | - Ensure code is conformant to the standard and passes test vectors |
| | 283 | - Ensure code does not attempt any invalid read or write using valgrind |
| | 284 | - Simple C-level optimization to get comparable speed with other crypto libraries |
| | 285 | |
| | 286 | '''Prerequisites:''' |
| | 287 | C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization. |
| | 288 | |
| | 289 | '''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task. |
| | 290 | |
| | 291 | '''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC) |
| | 292 | |
| | 293 | '''Backup Mentor:''' Michael Niedermayer (''michaelni'' in #ffmpeg-devel on Freenode IRC) |
| | 294 | |
| | 295 | |
| | 296 | == VDPAU filter == |
| | 297 | |
| | 298 | '''Description:''' VDPAU is not only about hardware-accelerated decoding but also allows some postprocessing – most notably deinterlacing. Other features of the API include scaling, noise removal, and a sharpening filter. |
| | 299 | |
| | 300 | '''Expected results:''' Implement a filter that allows usage of the postprocessing features as defined by the VDPAU API. This should particularly include the deinterlacer, and ideally all features should be usable to allow comparing the quality and performance of different hardware and hardware vs. software. |
| | 301 | |
| | 302 | '''Prerequisites:''' C coding skills, and you will need hardware that allows VDPAU post-processing. |
| | 303 | |
| | 304 | '''Qualification Task:''' Show that you are familiar with the FFmpeg filter system, either by porting one or more of the remaining postprocessing filters from the MPlayer compatibility layer (files `vf_*pp*.c` in `libavfilter/libmpcodecs`, see for comparison commit [http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=a2c547ff a2c547ff]) to a native filter, or by implementing a very simple inverse telecine filter under LGPL that allows reversal of the effect of the telecine filter. |
| | 305 | |
| | 306 | '''Mentor:''' Carl Eugen Hoyos (''cehoyos'' in #ffmpeg-devel on Freenode IRC) |
| | 307 | |
| | 308 | '''Backup Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, Reimar.Doeffinger [at] gmx.de) |
| | 309 | |
| | 310 | |
| | 311 | == Basic servers for network protocols == |
| | 312 | |
| | 313 | '''Description:''' libavformat contains clients for various network protocols used in multimedia streaming: HTTP, RTMP, MMS, RTSP. Your work will be to implement the server side for one or several of these protocols. |
| | 314 | |
| | 315 | The libavformat framework is not designed to build general-purpose server applications with several clients, and nothing similar to the configuration features of real servers like Apache is expected, but libavformat should be able to stream a single predefined bytestream to/from a single client. |
| | 316 | |
| | 317 | Note: server support is already implemented for the receiving side of RTSP. |
| | 318 | |
| | 319 | '''Expected results:''' basic servers for network protocols capable of interoperating with third-party clients. |
| | 320 | |
| | 321 | '''Prerequisites:''' C coding skills, basic familiarity with git, network programming. |
| | 322 | |
| | 323 | '''Qualification Task:''' proof-of-concept server for one of the protocol, capable of interacting with a particular client in controlled circumstances; or anything network-related, e.g. fixing a ticket in our [https://trac.ffmpeg.org/ bug tracker]. |
| | 324 | |
| | 325 | '''Mentor:''' Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC) |
| | 326 | |
| | 327 | '''Backup mentor:''' Reynaldo Verdejo (''reynaldo'' in #ffmpeg-devel on Freenode IRC, R Verdejo on g mail) |
| | 328 | |
| | 329 | ---- |
| | 330 | |
| | 331 | = Unmentored Projects = |
| | 332 | |
| | 333 | This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [#YourOwnProjectIdea propose their own project]. |
| 141 | | '''Qualification Task''': write or port one or more filters |
| 142 | | |
| 143 | | '''Mentor:''' TBA possibly Stefano Sabatini (saste at #ffmpeg-devel IRC) |
| 144 | | |
| 145 | | '''Backup mentor:''' Clément Bœsch (ubitux at #ffmpeg-devel IRC) |
| 146 | | |
| 147 | | == Subtitles == |
| 148 | | |
| 149 | | [[Image(subtitles_support.jpg, 300, right, nolink)]] |
| 150 | | |
| 151 | | '''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form. |
| 152 | | |
| 153 | | '''Expected results:''' |
| 154 | | |
| 155 | | - Add support for new subtitles formats. Example: a demuxer for .SUP files, just like !VobSub but for Blu-Ray, or a !VobSub muxer. |
| 156 | | - Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT. |
| 157 | | - Update the API to get rid of the clumsy internal text representation of styles |
| 158 | | - Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay. |
| 159 | | - BONUS: if everything goes well, the student will be allowed to add basic support for teletext |
| 160 | | |
| 161 | | '''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required. |
| 162 | | |
| 163 | | '''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood. |
| 164 | | |
| 165 | | '''Mentor:''' Clément Bœsch (ubitux at #ffmpeg-devel IRC) |
| 166 | | |
| 167 | | '''Backup Mentor:''' TBA, possibly Nicolas George (Cigaes at #ffmpeg-devel IRC) |
| 168 | | |
| 169 | | == Postproc optimizations == |
| 170 | | |
| 171 | | [[Image(PostProc.jpg, 240, right, nolink)]] |
| 172 | | |
| 173 | | '''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+). |
| 174 | | |
| 175 | | '''Expected results:''' |
| 176 | | |
| 177 | | - Convert all gcc inline asm in libpostproc to YASM. |
| 178 | | - Restructure the code so that it works with block sizes compatible with modern SIMD. |
| 179 | | - Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc. |
| 180 | | |
| 181 | | '''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git. |
| 182 | | |
| 183 | | '''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2. |
| 184 | | |
| 185 | | '''Mentor:''' Michael Niedermayer (michaelni at #ffmpeg-devel IRC) |
| 186 | | |
| 187 | | '''Backup Mentor:''' TBA, possibly Stefano Sabatini (saste at #ffmpeg-devel IRC) |
| 188 | | |
| 189 | | |
| 190 | | == Bayer RGB colorspaces == |
| 191 | | |
| 192 | | [[Image(bayer_rgb_colorspaces.png, right, nolink)]] |
| 193 | | |
| 194 | | '''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing. |
| 195 | | |
| 196 | | '''Expected Results:''' |
| 197 | | |
| 198 | | - Rebase existing patches |
| 199 | | - Implement high quality bayer transformations in libswscale (plain C) |
| 200 | | - Add bayer formats to the libavutil pixfmt enumeration routines |
| 201 | | - SIMD optimizations of the libswscale transformations |
| 202 | | - Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing) |
| 203 | | |
| 204 | | Optional goodies: |
| 205 | | |
| 206 | | - Extend TIFF decoder to support DNG-Bayer format |
| 207 | | - Support a popular proprietary camera format (many to choose from; see dcraw project) |
| 208 | | |
| 209 | | '''Prerequisites:''' C coding skills, familiarity with git/source code control systems. |
| 210 | | |
| 211 | | '''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale |
| 212 | | |
| 213 | | '''Mentor:''' TBA, possibly Peter Ross (pross-au at #ffmpeg-devel IRC) |
| 214 | | |
| 215 | | '''Backup Mentor:''' Michael Niedermayer (michaelni at #ffmpeg-devel IRC) |
| 216 | | |
| 217 | | == MPEG-4 ALS encoder == |
| 218 | | |
| 219 | | '''Description:''' A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on github. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the reference codec and specifications. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project. |
| 220 | | |
| 221 | | '''Expected results:''' |
| 222 | | |
| 223 | | - Update the existing encoder to fit into the current codebase. |
| 224 | | - Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE. |
| 225 | | - Ensure the FFmpeg decoder processes all generated files without warnings. |
| 226 | | - Enhance the rudimentary feature set of the encoder. |
| 227 | | |
| 228 | | '''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial. |
| 229 | | |
| 230 | | '''Qualification Task''': Add floating point support to MPEG-4 ALS decoder |
| 231 | | |
| 232 | | '''Mentor:''' Paul B Mahol (durandal_1707 at #ffmpeg-devel IRC) |
| 233 | | |
| 234 | | '''Backup Mentor:''' TBA, possibly Stefano Sabatini (saste at #ffmpeg-devel IRC) |
| 235 | | |
| 236 | | |
| 237 | | == Hardware Acceleration (hwaccel) API v2 == |
| 238 | | |
| 239 | | [[Image(hwaccel.jpg, right, nolink)]] |
| 240 | | |
| 241 | | '''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification. |
| 242 | | |
| 243 | | The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the !PixelFormats namespace. |
| 244 | | |
| 245 | | '''Expected results:''' |
| 246 | | |
| 247 | | - FFmpeg core library (libavcodec): |
| 248 | | - Core API extensions and improvements |
| 249 | | - Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications |
| 250 | | - Add new tokens describing hardware accelerators |
| 251 | | - Add new flags exposing HW capabilities like download/upload |
| 252 | | - Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers |
| 253 | | - Port hwaccels to v2 infrastructure |
| 254 | | - Port VA-API decoders to v2 infrastructure |
| 255 | | - Validate that VA-API decoders still work with existing applications supporting hwaccel v1 |
| 256 | | - Provide download capability through vaGetImage() |
| 257 | | - Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer |
| 258 | | - Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications |
| 259 | | |
| 260 | | - FFmpeg applications: |
| 261 | | - Integrate hardware acceleration into ffplay |
| 262 | | - Create a video-output (VO) infrastructure to ffplay |
| 263 | | - Port the SDL renderer to the new VO infrastructure |
| 264 | | - Add support for VA-API: VA renderer through vaPutSurface(), add -hwaccel option to select "vaapi" renderer |
| 265 | | - Add support for VDPAU (optional): VDPAU renderer through !VdpPresentationQueueDisplay() |
| 266 | | - Integrate hardware acceleration into ffmpeg |
| 267 | | - Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use |
| 268 | | |
| 269 | | '''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API. |
| 270 | | |
| 271 | | '''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc. |
| 272 | | |
| 273 | | '''Mentor:''' TBA, possibly Gwenole Beauchesne (!__gb!__ at #ffmpeg-devel IRC) |
| 274 | | |
| 275 | | '''Backup Mentor:''' TBA |
| | 354 | '''Qualification Task:''' write or port one or more filters |
| | 355 | |
| | 356 | '''Mentor:''' TBA |
| | 357 | |
| | 358 | '''Backup mentor:''' Clément Bœsch (''ubitux'' in #ffmpeg-devel on Freenode IRC) |
| | 359 | |
| | 360 | |
| | 361 | == TrueHD encoder == |
| | 362 | |
| | 363 | '''Description:''' FFmpeg currently does not support encoding to TrueHD, one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding. |
| | 364 | |
| | 365 | '''Expected results:''' a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding with a competetive compression rate |
| | 366 | |
| | 367 | '''Prerequisites:''' C coding skills, basic familiarity with git |
| | 368 | |
| | 369 | '''Qualification Task:''' write a encoder that produces a valid bitstream that can be decoded without errors to silence. |
| | 370 | |
| | 371 | == Opus decoder == |
| | 372 | |
| | 373 | [[Image(opus.png, right, nolink)]] |
| | 374 | |
| | 375 | '''Description:''' Opus decoding is currently supported only through the external libopus library |
| | 376 | |
| | 377 | '''Expected results:''' a native opus decoder in ffmpeg |
| | 378 | |
| | 379 | '''Prerequisites:''' C coding skills, basic familiarity with git, some knowledge about opus would be benefical |
| | 380 | |
| | 381 | '''Qualification Task:''' write part of the decoder, like write the bitstream decoding or header decoding and show that it is working. |
| | 382 | |
| | 383 | |
| | 384 | == VC-1 interlaced == |
| | 385 | |
| | 386 | '''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts. This task is difficult. |
| | 387 | |
| | 388 | '''Expected results:''' |
| | 389 | * Implement missing parts of interlacing |
| | 390 | * Make more reference samples bit-exact |
| | 391 | |
| | 392 | '''Prerequisites:''' C coding skills, basic familiarity with git |
| | 393 | |
| | 394 | '''Qualification Task:''' Find a bug in the current decoder implementation and fix it. |
| | 395 | |
| | 396 | |
| | 397 | == JPEG 2000 == |
| | 398 | |
| | 399 | '''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples. This task is difficult |
| | 400 | |
| | 401 | '''Expected results:''' A jpeg 2000 decoder which can decode most lossy samples without artifacts and lossless samples exactly. |
| | 402 | |
| | 403 | '''Qualification Task:''' Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation. |
| | 404 | |
| | 405 | '''Prerequisites:''' C coding skills, basic familiarity with git, good knowledge of Jpeg-2000 |
| 310 | | == AAC Improvements == |
| 311 | | |
| 312 | | '''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work. |
| 313 | | |
| 314 | | - AAC LD decoder |
| 315 | | - AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples |
| 316 | | - AAC SSR decoder |
| 317 | | - AAC 960/120 MDCT window |
| 318 | | - AAC multi-channel encoding |
| 319 | | |
| 320 | | '''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks. |
| 321 | | |
| 322 | | '''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful. |
| 323 | | |
| 324 | | '''Mentor:''' TBA, possibly Baptiste Coudurier (bcoudurier at #ffmpeg-devel IRC) |
| 325 | | |
| 326 | | '''Backup Mentor:''' TBA, possibly Stefano Sabatini (saste at #ffmpeg-devel IRC) |
| 327 | | |
| 328 | | |
| 329 | | == DTS / DCA Improvements == |
| 330 | | |
| 331 | | '''Description:''' FFmpeg contains a DTS decoder. |
| 332 | | |
| 333 | | - DTS-HD decoder improvements: A possible qualification task is to implement ticket #1920 |
| 334 | | - Add support for X96 extension (96khz) |
| 335 | | - Add support for XLL extension (lossless) |
| 336 | | - Add support for pure DTS-HD streams that do not contain a DTS core |
| 337 | | - Add support for multiple assets |
| 338 | | - Add support for LBR extension |
| 339 | | |
| 340 | | '''Prerequisites:''' C coding skills, basic familiarity with git. |
| 341 | | |
| 342 | | '''Mentor:''' TBA, possibly Benjamin Larsson (merbanan at #ffmpeg-devel IRC) |
| 343 | | |
| 344 | | '''Backup Mentor:''' TBA, possibly Stefano Sabatini (saste at #ffmpeg-devel IRC) |
| 345 | | |
| 346 | | == MXF Demuxer Improvements == |
| 347 | | |
| 348 | | '''Description:''' The MXF demuxer needs a proper, compact way to map !EssenceContainer ULs to !WrappingKind. See ticket #2776. Ticket #1916 also has relevant information. |
| 349 | | |
| 350 | | Essence in MXF is typically stored in one of two ways: as an audio/video interleave, or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically incorrect, but have worked due to a lack of samples demonstrating the contrary. |
| 351 | | |
| 352 | | '''Expected results:''' The sample in ticket #2776 should demux correctly, and add a test case in FATE. The solution should increase the size of libavformat by no more than 32 KiB. |
| 353 | | |
| 354 | | '''Prerequisites:''' C coding skills, basic familiarity with git. |
| 355 | | |
| 356 | | '''Qualification Task''': Investigate if there may be a compact way of representing the UL -> !WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 document]. The tables take up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat. |
| 357 | | |
| 358 | | '''Mentor:''' TBA, possibly Tomas Härdin (thardin at #ffmpeg-devel IRC) |
| 359 | | |
| 360 | | '''Backup Mentor:''' TBA |
| 361 | | |
| 362 | | ---- |
| 363 | | |
| 364 | | = Unmentored Projects = |
| 365 | | |
| 366 | | This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [#YourOwnProjectIdea propose their own project]. |
| | 440 | == H.264 Multiview Video Coding (MVC) == |
| | 441 | |
| | 442 | [[Image(h264_mvc.jpg, right, nolink)]] |
| | 443 | |
| | 444 | '''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the ffmpeg-devel mailing list. |
| | 445 | |
| | 446 | '''Expected results:''' Create MVC decoder and add a test for [http://ffmpeg.org/fate.html FATE]. |
| | 447 | |
| | 448 | '''Prerequisites:''' C coding skills, familiarity with git/source code control systems. |
| | 449 | |
| | 450 | '''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation. |
| | 451 | |
| | 452 | '''Mentor:''' TBA |
| | 453 | |
| | 454 | '''Backup mentor:''' TBA |
