Integrating Degradation Learning into Image Restoration

Existing image restoration methods usually assume specific degradation model, e.g., linear combination of clean image and degradation map in image denoising and deraining. Benefiting from the power of deep learning, a restoration mapping can be learned from degraded image to latent clean image. In this paper, we propose to integrate degradation learning into image restoration (IDLIR), where degradation model can be learned from training samples. In particular, IDLIR is an iterative restoration framework, where latent clean image and degradation map can be extracted from current residual degraded image, and are then fused by a degradation network to reconstruct degraded image. Then the residual degradation image can be updated by computing the difference between input and reconstructed degraded images. By taking denoising and deraining as examples, IDLIR is compared with state-of-the-art methods on several benchmark datasets. IDLIR performs better than state-of-the-art methods quantitatively and qualitatively.

[1]  J. Zhang,et al.  HINet: Half Instance Normalization Network for Image Restoration , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[2]  Ling Shao,et al.  Multi-Stage Progressive Image Restoration , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Cong Wang,et al.  DCSFN: Deep Cross-scale Fusion Network for Single Image Rain Removal , 2020, ACM Multimedia.

[4]  Deyu Meng,et al.  Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generation , 2020, ECCV.

[5]  Cong Wang,et al.  Physical Model Guided Deep Image Deraining , 2020, 2020 IEEE International Conference on Multimedia and Expo (ICME).

[6]  Chen Chen,et al.  Multi-Scale Progressive Fusion Network for Single Image Deraining , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Fahad Shahbaz Khan,et al.  CycleISP: Real Image Restoration via Improved Data Synthesis , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Yoonsik Kim,et al.  Transfer Learning From Synthetic to Real-Noise Denoising With Adaptive Instance Normalization , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Zhihai Xu,et al.  Spatial-Adaptive Network for Single Image Denoising , 2020, ECCV.

[10]  Deyu Meng,et al.  Variational Denoising Network: Toward Blind Noise Modeling and Removal , 2019, NeurIPS.

[11]  Vishal M. Patel,et al.  Uncertainty Guided Multi-Scale Residual Learning-Using a Cycle Spinning CNN for Single Image De-Raining , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Nick Barnes,et al.  Real Image Denoising With Feature Attention , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[13]  Qinghua Hu,et al.  Progressive Image Deraining Networks: A Better and Simpler Baseline , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Yinglong Wang,et al.  Removing Rain Streaks by a Linear Model , 2018, IEEE Access.

[15]  Ying Wu,et al.  Semi-Supervised Transfer Learning for Image Rain Removal , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Hongbin Zha,et al.  Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining , 2018, ECCV.

[17]  Wangmeng Zuo,et al.  Toward Convolutional Blind Denoising of Real Photographs , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Stephen Lin,et al.  A High-Quality Denoising Dataset for Smartphone Cameras , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Vishal M. Patel,et al.  Density-Aware Single Image De-raining Using a Multi-stream Dense Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Stefan Roth,et al.  Benchmarking Denoising Algorithms with Real Photographs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Delu Zeng,et al.  Removing Rain from Single Images via a Deep Detail Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Bing Zeng,et al.  A Hierarchical Approach for Rain or Snow Removing in a Single Color Image , 2017, IEEE Transactions on Image Processing.

[23]  Vishal M. Patel,et al.  Image De-Raining Using a Conditional Generative Adversarial Network , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Shuicheng Yan,et al.  Deep Joint Rain Detection and Removal from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[27]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Yu Luo,et al.  Removing Rain from a Single Image via Discriminative Sparse Coding , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[29]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[30]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Yu-Hsiang Fu,et al.  Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition , 2012, IEEE Transactions on Image Processing.

[32]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[33]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.