Iterative Residual Network for Deep Joint Image Demosaicking and Denoising
Modern digital cameras rely on sequential execution of separate image processing steps to produce realistic images. The first two steps are usually related to denoising and demosaicking where the former aims to reduce noise from the sensor and the latter converts a series of light intensity readings to color images. Modern approaches try to jointly solve these problems, i.e joint denoising-demosaicking which is an inherently ill-posed problem given that two-thirds of the intensity information are missing and the rest are perturbed by noise. While there are several machine learning systems that have been recently introduced to solve this problem, in this work we propose a novel algorithm which is inspired by powerful classical image regularization methods, large-scale optimization and deep learning techniques. Consequently, our derived neural network has a transparent and clear interpretation compared to other black-box data driven approaches. Our extensive experimentation line demonstrates that our proposed network outperforms any previous approaches on both noisy and noise-free data across many different datasets. This improvement in reconstruction quality is attributed to the principled way we design our network architecture, which as a result requires fewer trainable parameters than the current state-of-the art solution and furthermore can be efficiently trained by using a significantly smaller number of training data than existing deep demosaicking networks.
Get the code & data
- Get code from https://github.com/cig-skoltech/deep_demosaick
- Download data from https://drive.google.com/file/d/1DBc6Sul5vfnbP2dnx-51hqMTTVvEBAZO/view?usp=sharing. MIT dataset is not included, download seperately from here.
- Use Application.ipynb to demosaick real RAW images using one of the pre-trained models.
- Training instructions can be found on ReadMe.md
- Kokkinos, F., & Lefkimmiatis, S. (2018). Iterative Residual Network for Deep Joint Image Demosaicking and Denoising. arXiv preprint arXiv:1807.06403. (paper)
- Kokkinos, F., & Lefkimmiatis, S. (2018). Deep Image Demosaicking using a Cascade of Convolutional Residual Denoising Networks. The European Conference on Computer Vision (ECCV), 2018. (paper)
- Poster presented at ECCV 2018 Munich can be found here