Super Resolution Filter
Description
Using machine learning Super Resolution (SR) techniques implement an upscaling filter.
This project expects the student to research currently uses of SR techniques in production and state of the art methods.
The student will evaluate various methods and choose one that fits best the quality/performance requeriments.
Requeriments
- Good C skills
- Basic Git knowledge
- Good machine learning background, expecifically (convolutional) neural networks
- Familiarity with at least one machine learning framework (eg. TensorFlow)
Expected results
A ffmpeg filter which performs upscale
- The filter may accept pre-trained data
- The filter may be trained to adapt to specific content
Proposals
Proposals must show that the student have a basic understanding of the problem and a few techniques used.
There must be a basic schedule containing a sketch of the techniques that will be evaluated and the evaluation process.
On the project course the proposal schedule/goals may change but the schedule must show the student have a solid plan to be successful.
Getting started
- Read the developer guide
- Subscribe to the ffmpeg-devel list
- Clone the ffmpeg git repository
- Build the source
Qualification Task
Implement a ffmpeg filter which performs the SRCNN 2x upscale, pre-trained weights will be provided.
The model used can be the 3 layer CNN with 9-1-5 filter size.
You can get a glimpse of how to write a ffmpeg filter looking at the code in src/libavfilter/vf_*.c
Literature
Following there is some literature on the subject.
The student does not need to be limited to these papers, bringing more resources will positively affect your proposal.
- SRCNN - https://arxiv.org/abs/1501.00092
- Sub Pixel CNN - https://arxiv.org/abs/1609.05158
- Image Transform Net - https://arxiv.org/abs/1603.08155
- SRResNet/SRGAN - https://arxiv.org/abs/1609.04802
- A+ - http://www.vision.ee.ethz.ch/~timofter/publications/Timofte-ACCV-2014.pdf
