Image Denoising with Kernels Based on Natural Image Relations

A successful class of image denoising methods is based on Bayesian approaches working in wavelet representations. The performance of these methods improves when relations among the local frequency coefficients are explicitly included. However, in these techniques, analytical estimates can be obtained only for particular combinations of analytical models of signal and noise, thus precluding its straightforward extension to deal with other arbitrary noise sources. In this paper, we propose an alternative non-explicit way to take into account the relations among natural image wavelet coefficients for denoising: we use support vector regression (SVR) in the wavelet domain to enforce these relations in the estimated signal. Since relations among the coefficients are specific to the signal, the regularization property of SVR is exploited to remove the noise, which does not share this feature. The specific signal relations are encoded in an anisotropic kernel obtained from mutual information measures computed on a representative image database. In the proposed scheme, training considers minimizing the Kullback-Leibler divergence (KLD) between the estimated and actual probability functions (or histograms) of signal and noise in order to enforce similarity up to the higher (computationally estimable) order. Due to its non-parametric nature, the method can eventually cope with different noise sources without the need of an explicit re-formulation, as it is strictly necessary under parametric Bayesian formalisms. Results under several noise levels and noise sources show that: (1) the proposed method outperforms conventional wavelet methods that assume coefficient independence, (2) it is similar to state-of-the-art methods that do explicitly include these relations when the noise source is Gaussian, and (3) it gives better numerical and visual performance when more complex, realistic noise sources are considered. Therefore, the proposed machine learning approach can be seen as a more flexible (model-free) alternative to the explicit description of wavelet coefficient relations for image denoising.

[1]  Michael W. Marcellin,et al.  JPEG2000 - image compression fundamentals, standards and practice , 2002, The Kluwer International Series in Engineering and Computer Science.

[2]  Robert D. Nowak,et al.  Wavelet-based image estimation: an empirical Bayes approach using Jeffrey's noninformative prior , 2001, IEEE Trans. Image Process..

[3]  J.T. Kwok,et al.  Linear Dependency betweenand the Input Noise in -Support Vector Regression , 2001 .

[4]  Gustavo Camps-Valls,et al.  Perceptual adaptive insensitivity for support vector machine image coding , 2005, IEEE Transactions on Neural Networks.

[5]  Tony Jebara,et al.  Probability Product Kernels , 2004, J. Mach. Learn. Res..

[6]  Gustavo Camps-Valls,et al.  On the Suitable Domain for SVM Training in Image Coding , 2008, J. Mach. Learn. Res..

[7]  J. Malo,et al.  V1 non-linear properties emerge from local-to-global non-linear ICA , 2006, Network.

[8]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[9]  V. Kecman,et al.  Iterative Single Data Algorithm for Training Kernel Machines from Huge Data Sets: Theory and Performance , 2005 .

[10]  Antonio Artés-Rodríguez,et al.  A Profile-Dependent Kernel-Based Regression for Cyclosporine Concentration Prediction , 2001 .

[11]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[12]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[13]  M. Bertero,et al.  Ill-posed problems in early vision , 1988, Proc. IEEE.

[14]  LaparraValero,et al.  Image Denoising with Kernels Based on Natural Image Relations , 2010 .

[15]  Jinwen Tian,et al.  Wavelet domain image denoising via support vector regression , 2004 .

[16]  A. Barducci,et al.  Analysis and rejection of systematic disturbances in hyperspectral remotely sensed images of the Earth. , 2001, Applied optics.

[17]  Fernando Pérez-Cruz,et al.  Weighted least squares training of support vector classifiers leading to compact and adaptive schemes , 2001, IEEE Trans. Neural Networks.

[18]  Eero P. Simoncelli,et al.  A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients , 2000, International Journal of Computer Vision.

[19]  Bernhard Schölkopf,et al.  Experimentally optimal v in support vector regression for different noise models and parameter settings , 2004, Neural Networks.

[20]  Henry Stark,et al.  Probability, Random Processes, and Estimation Theory for Engineers , 1995 .

[21]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[22]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[23]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[24]  Yunqian Ma,et al.  Practical selection of SVM parameters and noise estimation for SVM regression , 2004, Neural Networks.

[25]  Yunqian Ma,et al.  Comparison of Model Selection for Regression , 2003, Neural Computation.

[26]  P Mouroulis,et al.  Design of pushbroom imaging spectrometers for optimum recovery of spectroscopic and spatial information. , 2000, Applied optics.

[27]  William T. Freeman,et al.  Presented at: 2nd Annual IEEE International Conference on Image , 1995 .

[28]  Ivor W. Tsang,et al.  Core Vector Machines: Fast SVM Training on Very Large Data Sets , 2005, J. Mach. Learn. Res..

[29]  Jia Jie Bayesian denoising of visual images in the wavelet domain , 2003 .

[30]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[31]  Aleksandra Pizurica,et al.  Removal of Correlated Noise by Modeling the Signal of Interest in the Wavelet Domain , 2009, IEEE Transactions on Image Processing.

[32]  B. R. Hunt,et al.  Digital Image Restoration , 1977 .

[33]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Francesc J. Ferri,et al.  Regularization operators for natural images based on nonlinear perception models , 2006, IEEE Transactions on Image Processing.

[35]  D. Donoho,et al.  Translation-Invariant De-Noising , 1995 .

[36]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[37]  Nick G. Kingsbury,et al.  Rotation-invariant local feature matching with complex wavelets , 2006, 2006 14th European Signal Processing Conference.

[38]  Tong Lee,et al.  Image approximation and smoothing by support vector regression , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[39]  Eero P. Simoncelli,et al.  Image compression via joint statistical characterization in the wavelet domain , 1999, IEEE Trans. Image Process..

[40]  J. Boulanger,et al.  Local adaptivity to variable smoothness for exemplar-based image denoising and representation , 2005 .

[41]  Jason Weston,et al.  Semi-supervised Protein Classification Using Cluster Kernels , 2003, NIPS.

[42]  Aapo Hyvärinen,et al.  Sparse Code Shrinkage: Denoising of Nongaussian Data by Maximum Likelihood Estimation , 1999, Neural Computation.

[43]  Virginia Torczon,et al.  On the Convergence of Pattern Search Algorithms , 1997, SIAM J. Optim..

[44]  Eero P. Simoncelli Statistical models for images: compression, restoration and synthesis , 1997, Conference Record of the Thirty-First Asilomar Conference on Signals, Systems and Computers (Cat. No.97CB36136).

[45]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[46]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[47]  R. R. Clarke Transform coding of images , 1985 .

[48]  Bram van Ginneken,et al.  Image Denoising with k-nearest Neighbor and Support Vector Regression , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[49]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[50]  Vojislav Kecman,et al.  Bias Term b in SVMs Again , 2004, ESANN.

[51]  Eero P. Simoncelli,et al.  Nonlinear image representation for efficient perceptual coding , 2006, IEEE Transactions on Image Processing.

[52]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[53]  Edward H. Adelson,et al.  Pyramid as a Compact Image Code , 1983 .

[54]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[55]  J. Mercer Functions of positive and negative type, and their connection with the theory of integral equations , 1909 .

[56]  Valero Laparra,et al.  Divisive normalization image quality metric revisited. , 2010, Journal of the Optical Society of America. A, Optics, image science, and vision.

[57]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[58]  Robert Michael Lewis,et al.  A Globally Convergent Augmented Lagrangian Pattern Search Algorithm for Optimization with General Constraints and Simple Bounds , 2002, SIAM J. Optim..

[59]  Pierre Moulin,et al.  Information-theoretic analysis of interscale and intrascale dependencies between image wavelet coefficients , 2001, IEEE Trans. Image Process..

[60]  Alexander J. Smola,et al.  Step Size Adaptation in Reproducing Kernel Hilbert Space , 2006, J. Mach. Learn. Res..

[61]  Fernando Pérez-Cruz,et al.  An IRWLS procedure for SVR , 2000, 2000 10th European Signal Processing Conference.