Changes between Version 8 and Version 9 of SponsoringPrograms/GSoC/2018/Results


Ignore:
Timestamp:
Aug 9, 2018, 4:32:28 PM (10 months ago)
Author:
HighVoltage
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • SponsoringPrograms/GSoC/2018/Results

    v8 v9  
    3535''Corrected by Weighted Grey Edge:''[[BR]]
    3636[[Image(https://firebasestorage.googleapis.com/v0/b/ffmpeg-color-constancy.appspot.com/o/weighted_grey_edge.png?alt=media&token=3b4047a4-04c2-4b48-8176-7905f4e282b4)]]
     37
     38== Super Resolution filter ==
     39
     40'''Description:''' Super resolution methods aim to recover high-resolution images from low-resolution images. These methods find direct application in many areas such as medical imaging, security and surveillance imaging, HDTV, satellite imaging. This project goal was to implement machine-learning-based super-resolution upscaling filter.
     41
     42'''Results:''' The basis of this project was the result of the qualification task implementing [[https://arxiv.org/abs/1501.00092 | SRCNN]] model for image super-resolution ([[https://github.com/FFmpeg/FFmpeg/commit/9479955c626529550d337067af85760e256eabb3 | commit]]). Contributions of the GSoC period include:
     43* Introduction of DNN inference module ([[https://github.com/FFmpeg/FFmpeg/commit/bdf1bbdbb4ebb342c0267d0f77cd06e717197e65 | commit]]). This module includes interface for model loading and execution as well as simple backend, which supports layers required for the super resolution filter.
     44* Introduction of [[https://www.tensorflow.org | TensorFlow]] backend to DNN inference module ([[https://github.com/FFmpeg/FFmpeg/commit/d8c0bbb0aa45eed61b159c4842473fe27e77ac12 | commit]]). It implements backend executing binary [[https://www.tensorflow.org | TensorFlow]] models using [[https://www.tensorflow.org | TensorFlow]] framework.
     45* Implementation of [[https://arxiv.org/abs/1609.05158 | ESPCN]] model for super resolution ([[https://github.com/FFmpeg/FFmpeg/commit/575b7189908e1cfa55104b0d2c7c9f6ea30ca2dc | commit]]). This model shows better results in terms of restoration accuracy and speed.
     46* Various improvements and fixes of the DNN module and the super resolution filter. Commits: [[https://github.com/FFmpeg/FFmpeg/commit/d29c35b4d8f1ce93058697e9819fda8684928109 | 1]], [[https://github.com/FFmpeg/FFmpeg/commit/648361c2faf39dbd00b716367add6ccea1464030 | 2]], [[https://github.com/FFmpeg/FFmpeg/commit/4eb63efbdaea6d36ad94f1bb0dd129b7f7aaa899 | 3]], [[https://github.com/FFmpeg/FFmpeg/commit/9d87897ba84a3b639a4c3afeb4ec6d21bc306a92 | 4]], [[https://github.com/FFmpeg/FFmpeg/commit/a66e74306a36c8467452a67d2d1af51dbbb9b4c5 | 5]].
     47Scripts for model training and evaluation as well as generation of model files, suitable for the super resolution filter, are provided in the [[https://github.com/HighVoltageRocknRoll/sr | repository]]. In addition to [[https://arxiv.org/abs/1501.00092 | SRCNN]] and [[https://arxiv.org/abs/1609.05158 | ESPCN]] models, [[https://arxiv.org/abs/1611.05250 | VESPCN]] and [[https://ieeexplore.ieee.org/document/7444187/ | VSRNet]] models were trained and evaluated, but they were not implemented in the super resolution filter due to low increase in restoration accuracy.
     48
     49'''Future work:''' After the GSoC period resnet-based models will be evaluated. If their upscaling quality is good, one of them will be added to the super resolution filter.
     50
     51'''Mentor:''' Pedro Souza (bygrandao AT gmail DOT com)
     52
     53'''Student:''' Sergey Lavrushkin (dualfal AT gmail DOT com)