Recent developments in computational color image denoising with PDEs to deep learning: a review

Image denoising methods are of fundamental importance in image processing and artificial intelligence systems. In this review, we analyze the traditional and state of the art mathematical models for computational color image denoising. These algorithms are divided into methods that are based on the partial differential equations, low rank, sparse representation and recent developments based on deep learning models. These algorithms also compared in terms of image quality measures. Our analysis and review of the computational color image denoising filters indicate that the convolutional neural networks from the deep learning domain obtain high quality restorations in terms of image quality despite the higher computational complexity.

[1]  Domenec Puig,et al.  Edge-preserving color image denoising through tensor voting , 2011, Comput. Vis. Image Underst..

[2]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[4]  Jaakko Lehtinen,et al.  Noise2Noise: Learning Image Restoration without Clean Data , 2018, ICML.

[5]  Elena Braverman,et al.  Adaptive frame-based color image denoising☆ , 2016 .

[6]  Saeid Saryazdi,et al.  An adaptive diffusion coefficient selection for image denoising , 2017, Digit. Signal Process..

[7]  Hugo Proença,et al.  Robust Periocular Recognition by Fusing Sparse Representations of Color and Geometry Information , 2015, Journal of Signal Processing Systems.

[8]  Shan Gai,et al.  Color image denoising via monogenic matrix-based sparse representation , 2017, The Visual Computer.

[9]  Jean-Michel Morel,et al.  The Noise Clinic: a Blind Image Denoising Algorithm , 2015, Image Process. Line.

[10]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[11]  Wangmeng Zuo,et al.  Attention-guided CNN for image denoising , 2020, Neural Networks.

[12]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[13]  Shunli Wang,et al.  Detail retaining convolutional neural network for image denoising , 2020, J. Vis. Commun. Image Represent..

[14]  Konstantinos N. Plataniotis,et al.  Adaptive filters for color image processing: A survey , 2000, 2000 10th European Signal Processing Conference.

[15]  Chunwei Tian,et al.  Image denoising using deep CNN with batch renormalization , 2020, Neural Networks.

[16]  Mingxuan Sun,et al.  Dilated Residual Network for Image Denoising , 2017, ArXiv.

[17]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[18]  Ron Kimmel,et al.  A general framework for low level vision , 1998, IEEE Trans. Image Process..

[19]  Tony F. Chan,et al.  Total Variation Denoising and Enhancement of Color Images Based on the CB and HSV Color Models , 2001, J. Vis. Commun. Image Represent..

[20]  V. B. Surya Prasath,et al.  Multispectral image denoising by well-posed anisotropic diffusion scheme with channel coupling , 2010 .

[21]  Amlan Chakrabarti,et al.  A Brief Survey of Recent Edge-Preserving Smoothing Algorithms on Digital Images , 2015, ArXiv.

[22]  Freddie Åström,et al.  A Geometric Approach to Color Image Regularization , 2016, ArXiv.

[23]  V. B. Surya Prasath,et al.  On convergent finite difference schemes for variational–PDE-based image processing , 2013, ArXiv.

[24]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[25]  Etienne E. Kerre,et al.  Fuzzy Random Impulse Noise Removal From Color Image Sequences , 2011, IEEE Transactions on Image Processing.

[26]  Aline Roumy,et al.  Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding , 2012, BMVC.

[27]  Gunther Wyszecki,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd Edition , 2000 .

[28]  Stamatios Lefkimmiatis,et al.  Non-local Color Image Denoising with Convolutional Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Liangpei Zhang,et al.  Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[30]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[32]  Jayant Shah,et al.  A common framework for curve evolution, segmentation and anisotropic diffusion , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  Michael Elad,et al.  Sparse Representation for Color Image Restoration , 2008, IEEE Transactions on Image Processing.

[34]  Yong Man Ro,et al.  Local Color Vector Binary Patterns From Multichannel Face Images for Face Recognition , 2012, IEEE Transactions on Image Processing.

[35]  J. K. Mandal,et al.  A Fuzzy Switching Median Filter of Impulses in Digital Imagery (FSMF) , 2014, Circuits Syst. Signal Process..

[36]  Taesup Moon,et al.  Fully Convolutional Pixel Adaptive Image Denoiser , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[37]  Licheng Yu,et al.  Vector Sparse Representation of Color Image Using Quaternion Matrix Analysis , 2015, IEEE Transactions on Image Processing.

[38]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[39]  David Dagan Feng,et al.  Fuzzy vector partition filtering technique for color image restoration , 2007, Comput. Vis. Image Underst..

[40]  Jie Li,et al.  Hybrid Noise Removal in Hyperspectral Imagery With a Spatial–Spectral Gradient Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[42]  Lei Zhang,et al.  Color demosaicking by local directional interpolation and nonlocal adaptive thresholding , 2011, J. Electronic Imaging.

[43]  Licheng Yu,et al.  Quaternion-based sparse representation of color image , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[44]  Sunil Agrawal,et al.  Image denoising review: From classical to state-of-the-art approaches , 2020, Inf. Fusion.

[45]  David Tschumperlé,et al.  Fast Anisotropic Smoothing of Multi-Valued Images using Curvature-Preserving PDE's , 2006, International Journal of Computer Vision.

[46]  Liang Lin,et al.  Multi-level Wavelet-CNN for Image Restoration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[47]  Mohammed Hassan,et al.  Structural Similarity Measure for Color Images , 2012 .

[48]  Lei Zhang,et al.  FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising , 2017, IEEE Transactions on Image Processing.

[49]  Mingxuan Sun,et al.  Dilated Deep Residual Network for Image Denoising , 2017, 2017 IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI).

[50]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[51]  Tony F. Chan,et al.  Color TV: total variation methods for restoration of vector-valued images , 1998, IEEE Trans. Image Process..

[52]  Lei Zhang,et al.  Multispectral Images Denoising by Intrinsic Tensor Sparsity Regularization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  C. Boncelet.  Image Noise Models , 2009 .

[54]  Nahum Kiryati,et al.  Color Image Deblurring with Impulsive Noise , 2005, VLSM.

[55]  Jonathan Cheung-Wai Chan,et al.  Hyperspectral Image Denoising Using Global Weighted Tensor Norm Minimum and Nonlocal Low-Rank Approximation , 2019, Remote. Sens..

[56]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Stefan Roth,et al.  Benchmarking Denoising Algorithms with Real Photographs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[59]  V. B. Surya Prasath.  Weighted laplacian differences based multispectral anisotropic diffusion , 2011, 2011 IEEE International Geoscience and Remote Sensing Symposium.

[60]  Mi-Suen Lee,et al.  A Computational Framework for Segmentation and Grouping , 2000 .

[61]  Yunsong Li,et al.  Hyperspectral Imagery Denoising by Deep Learning With Trainable Nonlinearity Function , 2017, IEEE Geoscience and Remote Sensing Letters.

[62]  Alen Begovic,et al.  Application of Artificial Neural Network for Image Noise Level Estimation in the SVD domain , 2019, Electronics.

[63]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[64]  Tony F. Chan,et al.  Variational Restoration of Nonflat Image Features: Models and Algorithms , 2001, SIAM J. Appl. Math..

[65]  V. B. Surya Prasath,et al.  Weighted and well-balanced anisotropic diffusion scheme for image denoising and restoration , 2014 .

[66]  L. Ambrosio,et al.  Approximation of functional depending on jumps by elliptic functional via t-convergence , 1990 .

[67]  V. B. Surya Prasath,et al.  Color image processing by vectorial total variation with gradient channels coupling , 2016 .

[68]  Jean-Michel Morel,et al.  A review of P.D.E. models in image processing and image analysis , 2002 .

[69]  Tony F. Chan,et al.  The digital TV filter and nonlinear denoising , 2001, IEEE Trans. Image Process..

[70]  R. Deriche,et al.  Anisotropic Diffusion Partial Differential Equations for Multichannel Image Regularization: Framework and Applications , 2007 .

[71]  M. Luo,et al.  The development of the CIE 2000 Colour Difference Formula , 2001 .

[72]  Suk-Hwan Lee,et al.  A Deep Feature Extraction Method for HEp-2 Cell Image Classification , 2018, Electronics.

[73]  Seongjai Kim,et al.  PDE-based image restoration: a hybrid model and color image denoising , 2006, IEEE Transactions on Image Processing.

[74]  Zhiqiang Huang,et al.  An Adaptive Nonlocal Gaussian Prior for Hyperspectral Image Denoising , 2019, IEEE Geoscience and Remote Sensing Letters.

[75]  Zhiyu Lyu,et al.  A nonsubsampled countourlet transform based CNN for real image denoising , 2020, Signal Process. Image Commun..

[76]  Wangmeng Zuo,et al.  Toward Convolutional Blind Denoising of Real Photographs , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[77]  Feng Jiang,et al.  Hierarchical residual learning for image denoising , 2019, Signal Process. Image Commun..

[78]  Michael Elad,et al.  Image Denoising Via Learned Dictionaries and Sparse representation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[79]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[80]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[81]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[82]  Debin Zhao,et al.  Image super-resolution via dual-dictionary learning and sparse representation , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[83]  Qiang Zhang,et al.  Hyperspectral Image Denoising Employing a Spatial–Spectral Deep Residual Convolutional Neural Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[84]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[85]  Xiaowei Yang,et al.  Color Image and Multispectral Image Denoising Using Block Diagonal Representation , 2019, IEEE Transactions on Image Processing.

[86]  Peng Liu,et al.  Learning Pixel-Distribution Prior with Wider Convolution for Image Denoising , 2017, ArXiv.

[87]  Samuel Morillas,et al.  Perceptual similarity between color images using fuzzy metrics , 2016, J. Vis. Commun. Image Represent..

[88]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[89]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[90]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[91]  Tao Chen,et al.  Spectral-spatial adaptive and well-balanced flow-based anisotropic diffusion for multispectral image denoising , 2017, J. Vis. Commun. Image Represent..

[92]  J. Weickert,et al.  Convex regularization of multi-channel images based on variants of the TV-model , 2017, 1804.01324.