DIVA: Deep Unfolded Network from Quantum Interactive Patches for Image Restoration

This paper presents a deep neural network called DIVA unfolding a baseline adaptive denoising algorithm (De-QuIP), relying on the theory of quantum many-body physics. Furthermore, it is shown that with very slight modifications, this network can be enhanced to solve more challenging image restoration tasks such as image deblurring, super-resolution and inpainting. Despite a compact and interpretable (from a physical perspective) architecture, the proposed deep learning network outperforms several recent algorithms from the literature, designed specifically for each task. The key ingredients of the proposed method are on one hand, its ability to handle non-local image structures through the patch-interaction term and the quantum-based Hamiltonian operator, and, on the other hand, its flexibility to adapt the hyperparameters patch-wisely, due to the training process.

[1]  A. Basarab,et al.  Deep Unfolding of Image Denoising by Quantum Interactive Patches , 2022, 2022 IEEE International Conference on Image Processing (ICIP).

[2]  Nwigbo Kenule Tuador,et al.  Quantum Denoising-Based Super-Resolution Algorithm Applied to Dental Tomography Images , 2022, 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI).

[3]  Ce Zhu,et al.  Low-Rankness Guided Group Sparse Representation for Image Restoration. , 2022, IEEE transactions on neural networks and learning systems.

[4]  Jie Liu,et al.  DFAN: Dual Feature Aggregation Network for Lightweight Image Super-Resolution , 2022, Wireless Communications and Mobile Computing.

[5]  Xiangchu Feng,et al.  Deep RED Unfolding Network for Image Restoration , 2021, IEEE Transactions on Image Processing.

[6]  A. Basarab,et al.  A Novel Image Denoising Algorithm Using Concepts of Quantum Many-Body Theory , 2021, Signal Process..

[7]  Moncef Gabbouj,et al.  Image denoising by Super Neurons: Why go deep? , 2021, 2111.14948.

[8]  A. Basarab,et al.  Despeckling Ultrasound Images Using Quantum Many-Body Physics , 2021, 2021 IEEE International Ultrasonics Symposium (IUS).

[9]  Adrian Basarab,et al.  Image Denoising Inspired by Quantum Many-Body physics , 2021, 2021 IEEE International Conference on Image Processing (ICIP).

[10]  Adrian Basarab,et al.  Plug-and-Play Quantum Adaptive Denoiser for Deconvolving Poisson Noisy Images , 2021, IEEE Access.

[11]  Xin Yuan,et al.  Triply Complementary Priors for Image Restoration , 2021, IEEE Transactions on Image Processing.

[12]  Jianmin Bao,et al.  Uformer: A General U-Shaped Transformer for Image Restoration , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Stefan Roth,et al.  Deep Wiener Deconvolution: Wiener Meets Deep Learning for Image Deblurring , 2021, NeurIPS.

[14]  Adrian Basarab,et al.  Poisson Image Deconvolution by a Plug-and-Play Quantum Denoising Scheme , 2020, 2021 29th European Signal Processing Conference (EUSIPCO).

[15]  Yuhui Quan,et al.  Variational-EM-Based Deep Learning for Noise-Blind Image Deblurring , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Denis Kouam'e,et al.  Quantum Mechanics-Based Signal and Image Representation: Application to Denoising , 2020, IEEE Open Journal of Signal Processing.

[17]  Luc Van Gool,et al.  Deep Unfolding Network for Image Super-Resolution , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Haiyan Wang,et al.  Learning Combination of Graph Filters for Graph Signal Modeling , 2019, IEEE Signal Processing Letters.

[19]  Michael Elad,et al.  Deep K-SVD Denoising , 2019, IEEE Transactions on Image Processing.

[20]  Nick Barnes,et al.  Densely Residual Laplacian Super-Resolution , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Yun Fu,et al.  Residual Dense Network for Image Restoration , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Yonina C. Eldar,et al.  Deep Unfolded Robust PCA with Application to Clutter Suppression in Ultrasound , 2018, bioRxiv.

[23]  Vivek K Goyal,et al.  Quantum-inspired computational imaging , 2018, Science.

[24]  Kyung-Ah Sohn,et al.  Fast, Accurate, and, Lightweight Super-Resolution with Cascading Residual Network , 2018, ECCV.

[25]  Dacheng Tao,et al.  Training Very Deep CNNs for General Non-Blind Deconvolution , 2018, IEEE Transactions on Image Processing.

[26]  Guangming Shi,et al.  Denoising Prior Driven Deep Neural Network for Image Restoration , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Lei Zhang,et al.  FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising , 2017, IEEE Transactions on Image Processing.

[28]  Carsten Rother,et al.  Learning to Push the Limits of Efficient FFT-Based Image Deconvolution , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Jian Yang,et al.  MemNet: A Persistent Memory Network for Image Restoration , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  Eirikur Agustsson,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[31]  Seungyong Lee,et al.  Fast non-blind deconvolution via regularized residual networks with long/short skip-connections , 2017, 2017 IEEE International Conference on Computational Photography (ICCP).

[32]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Michael Unser,et al.  Deep Convolutional Neural Network for Inverse Problems in Imaging , 2016, IEEE Transactions on Image Processing.

[35]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[36]  Sivaram Prasad Mudunuri,et al.  Low Resolution Face Recognition Across Variations in Pose and Illumination , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  José M. Bioucas-Dias,et al.  Image restoration and reconstruction using variable splitting and class-adapted image priors , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[38]  Maurice Borgeaud,et al.  Kernel Low-Rank and Sparse Graph for Unsupervised and Semi-Supervised Classification of Hyperspectral Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Yunjin Chen,et al.  Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  A. El-Rafei,et al.  A quantum mechanics-based framework for image processing and its application to image segmentation , 2015, Quantum Information Processing.

[42]  Adrian Basarab,et al.  Compressive Deconvolution in Medical Ultrasound Imaging , 2015, IEEE Transactions on Medical Imaging.

[43]  Taous-Meriem Laleg-Kirati,et al.  A novel algorithm for image representation using discrete spectrum of the Schrödinger operator , 2015, Digit. Signal Process..

[44]  Marcelo Pereyra,et al.  Fast Unsupervised Bayesian Image Segmentation With Adaptive Spatial Regularisation , 2015, IEEE Transactions on Image Processing.

[45]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Stefan Roth,et al.  Shrinkage Fields for Effective Image Restoration , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Moncef Gabbouj,et al.  Quantum mechanics in computer vision: Automatic object extraction , 2013, 2013 IEEE International Conference on Image Processing.

[48]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[49]  Raymond H. Chan,et al.  Constrained Total Variation Deblurring Models and Fast Algorithms Based on Alternating Direction Method of Multipliers , 2013, SIAM J. Imaging Sci..

[50]  Guangming Shi,et al.  Nonlocal Image Restoration With Bilateral Variance Estimation: A Low-Rank Approach , 2013, IEEE Transactions on Image Processing.

[51]  Mário A. T. Figueiredo,et al.  Deconvolving Images With Unknown Boundaries Using the Alternating Direction Method of Multipliers , 2012, IEEE Transactions on Image Processing.

[52]  Xuelong Li,et al.  Image Super-Resolution With Sparse Neighbor Embedding , 2012, IEEE Transactions on Image Processing.

[53]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Karen O. Egiazarian,et al.  BM3D Frames and Variational Image Deblurring , 2011, IEEE Transactions on Image Processing.

[55]  Harry Shum,et al.  Gradient Profile Prior and Its Applications in Image Super-Resolution and Enhancement , 2011, IEEE Transactions on Image Processing.

[56]  Frédo Durand,et al.  Efficient marginal likelihood optimization in blind deconvolution , 2011, CVPR 2011.

[57]  Heung-Yeung Shum,et al.  Gradient Profile Prior and Its Applications in Image Super-Resolution and Enhancement , 2011, IEEE Transactions on Image Processing.

[58]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[59]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Yann LeCun,et al.  Learning Fast Approximations of Sparse Coding , 2010, ICML.

[61]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[62]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[63]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[64]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[65]  Scott T. Acton,et al.  Speckle reducing anisotropic diffusion , 2002, IEEE Trans. Image Process..

[66]  Alexei A. Efros,et al.  Fast bilateral filtering for the display of high-dynamic-range images , 2002 .

[67]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[68]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[69]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[70]  Jian Sun,et al.  BM3D-Net: A Convolutional Neural Network for Transform-Domain Collaborative Filtering , 2018, IEEE Signal Processing Letters.

[71]  A. Basarab,et al.  Fast Single Image Super-resolution using a New Analytical Solution for l2-l2 Problems. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[72]  Mário A. T. Figueiredo,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2010, IEEE Transactions on Image Processing.

[73]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[74]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[75]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .