Local Adaptivity to Variable Smoothness for Exemplar-Based Image Regularization and Representation

Abstract A novel adaptive and exemplar-based approach is proposed for image restoration (denoising) and representation. The method is based on a pointwise selection of similar image patches of fixed size in the variable neighborhood of each pixel. The main idea is to associate with each pixel the weighted sum of data points within an adaptive neighborhood. We use small image patches (e.g. 7×7 or 9×9 patches) to compute these weights since they are able to capture local geometric patterns and texels seen in images. In this paper, we mainly focus on the problem of adaptive neighborhood selection in a manner that balances the accuracy of approximation and the stochastic error, at each spatial position. The proposed pointwise estimator is then iterative and automatically adapts to the degree of underlying smoothness with minimal a priori assumptions on the function to be recovered. The method is applied to artificially corrupted real images and the performance is very close, and in some cases even surpasses, to that of the already published denoising methods. The proposed algorithm is demonstrated on real images corrupted by non-Gaussian noise and is used for applications in bio-imaging.

[1]  Luis Enrique Sucar,et al.  Probabilistic estimation of local scale , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[2]  Stéphane Mallat,et al.  Sparse geometric image representations with bandelets , 2005, IEEE Transactions on Image Processing.

[3]  S. Osher,et al.  IMAGE DECOMPOSITION AND RESTORATION USING TOTAL VARIATION MINIMIZATION AND THE H−1 NORM∗ , 2002 .

[4]  Guang-Zhong Yang,et al.  Structure adaptive anisotropic image filtering , 1996, Image Vis. Comput..

[5]  J. Simonoff.  Multivariate Density Estimation , 1996 .

[6]  Jong-Sen Lee,et al.  Digital image smoothing and the sigma filter , 1983, Comput. Vis. Graph. Image Process..

[7]  Michael Elad,et al.  Image Denoising Via Learned Dictionaries and Sparse representation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[8]  Jörg Weule,et al.  Non-Linear Gaussian Filters Performing Edge Preserving Diffusion , 1995, DAGM-Symposium.

[9]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[10]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[11]  Yves Meyer,et al.  Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures , 2001 .

[12]  Ron Kimmel,et al.  Efficient Beltrami Flow Using a Short Time Kernel , 2003, Scale-Space.

[13]  D. L. Donoho,et al.  Ideal spacial adaptation via wavelet shrinkage , 1994 .

[14]  Antonin Chambolle,et al.  Image Decomposition into a Bounded Variation Component and an Oscillating Component , 2005, Journal of Mathematical Imaging and Vision.

[15]  Guy Gilboa,et al.  Nonlocal Linear Image Regularization and Supervised Segmentation , 2007, Multiscale Model. Simul..

[16]  Jaakko Astola,et al.  Adaptive Window Size Image De-noising Based on Intersection of Confidence Intervals (ICI) Rule , 2002, Journal of Mathematical Imaging and Vision.

[17]  Charles Kervrann.  An Adaptive Window Approach for Image Smoothing and Structures Preserving , 2004, ECCV.

[18]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[19]  Wolfgang Härdle,et al.  APPLIED NONPARAMETRIC METHODS , 1992 .

[20]  N. Sochen,et al.  Texture Preserving Variational Denoising Using an Adaptive Fidelity Term , 2003 .

[21]  Alfred M. Bruckstein,et al.  Diffusions and Confusions in Signal and Image Processing , 2001, Journal of Mathematical Imaging and Vision.

[22]  Guillermo Sapiro,et al.  Edges as Outliers: Anisotropic Smoothing Using Local Image Statistics , 1999, Scale-Space.

[23]  David Zhang,et al.  Image information restoration based on long-range correlation , 2002, IEEE Trans. Circuits Syst. Video Technol..

[24]  Charles Kervrann,et al.  Unsupervised Patch-Based Image Regularization and Representation , 2006, ECCV.

[25]  David Tschumperlé,et al.  Curvature-Preserving regularization of multi-valued images using PDE's , 2006, eccv 2006.

[26]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Brendan J. Frey,et al.  Epitomic analysis of appearance and shape , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[29]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[30]  Aleksandra Pizurica,et al.  Estimating the probability of the presence of a signal of interest in multiresolution single- and multiband image denoising , 2006, IEEE Transactions on Image Processing.

[31]  Joachim Weickert,et al.  Coherence-Enhancing Diffusion Filtering , 1999, International Journal of Computer Vision.

[32]  Pierrick Coupé,et al.  Bayesian Non-local Means Filter, Image Redundancy and Adaptive Dictionaries for Noise Removal , 2007, SSVM.

[33]  E. Mammen,et al.  Optimal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selectors , 1997 .

[34]  Song-Chun Zhu.  Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling , 1998 .

[35]  Ljubisa Stankovic,et al.  Performance analysis of the adaptive algorithm for bias-to-variance trade-off , 2003 .

[36]  Vladimir Spokoiny,et al.  ESTIMATION OF A FUNCTION WITH DISCONTINUITIES VIA LOCAL POLYNOMIAL FIT WITH AN ADAPTIVE WINDOW CHOICE , 1998 .

[37]  William T. Freeman,et al.  Learning low-level vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[38]  Pavel Mrázek.  Selection of Optimal Stopping Time for Nonlinear Diffusion Filtering , 2001, Scale-Space.

[39]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[40]  David Zhang,et al.  Restoration of impulse noise corrupted images using long-range correlation , 1998, IEEE Signal Processing Letters.

[41]  O. Lepskii.  On a Problem of Adaptive Estimation in Gaussian White Noise , 1991 .

[42]  Stanley Osher,et al.  Deblurring and Denoising of Images by Nonlocal Functionals , 2005, Multiscale Model. Simul..

[43]  Murray Eden,et al.  Fundamentals of Digital Optics , 1996 .

[44]  Guillermo Sapiro,et al.  Robust anisotropic diffusion , 1998, IEEE Trans. Image Process..

[45]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[46]  Michael Elad,et al.  On the origin of the bilateral filter and ways to improve it , 2002, IEEE Trans. Image Process..

[47]  Peihua Qiu,et al.  Edge-preserving image denoising and estimation of discontinuous surfaces , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Samuel Audet,et al.  Image-Based Rendering Using Image-Based Priors , 2006 .

[49]  Stanley Osher,et al.  Image Decomposition and Restoration Using Total Variation Minimization and the H1 , 2003, Multiscale Model. Simul..

[50]  R. van den Boomgaard,et al.  On the equivalence of local-mode finding, robust estimation and mean-shift analysis as used in early vision tasks , 2002, Object recognition supported by user interaction for service robots.

[51]  Ron Kimmel,et al.  Optimal Algorithm for Shape from Shading and Path Planning , 2001, Journal of Mathematical Imaging and Vision.

[52]  Michael Elad,et al.  On the bilateral filter and ways to improve it , 2002 .

[53]  Narendra Ahuja,et al.  Regression based bandwidth selection for segmentation using Parzen windows , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[54]  Daniel Cremers,et al.  Iterated Nonlocal Means for Texture Restoration , 2007, SSVM.

[55]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[56]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[57]  Philippe Saint-Marc,et al.  Adaptive Smoothing: A General Tool for Early Vision , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[58]  J. Polzehl,et al.  Adaptive weights smoothing with applications to image restoration , 1998 .

[59]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[60]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[61]  Michael Elad,et al.  Multiscale Sparse Image Representation with Learned Dictionaries (PREPRINT) , 2007 .

[62]  Emmanuel J. Candès,et al.  The curvelet transform for image denoising , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[63]  Tony F. Chan,et al.  The digital TV filter and nonlinear denoising , 2001, IEEE Trans. Image Process..

[64]  Fred Godtliebsen,et al.  A nonlinear gaussian filter applied to images with discontinuities , 1997 .

[65]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[66]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[67]  Thomas Brox,et al.  A TV Flow Based Local Scale Measure for Texture Discrimination , 2004, ECCV.

[68]  Joost van de Weijer,et al.  Local Mode Filtering , 2001, CVPR.

[69]  Dorin Comaniciu,et al.  A common framework for nonlinear diffusion, adaptive smoothing, bilateral filtering and mean shift , 2004, Image Vis. Comput..

[70]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[71]  T. Gasser,et al.  Residual variance and residual pattern in nonlinear regression , 1986 .

[72]  J. Marron,et al.  Edge-Preserving Smoothers for Image Processing , 1998 .

[73]  Patrick Bouthemy,et al.  Space-Time Adaptation for Patch-Based Image Sequence Restoration , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[74]  Edward R. Vrscay,et al.  Fractal image denoising , 2003, IEEE Trans. Image Process..

[75]  J. Boulanger,et al.  Local adaptivity to variable smoothness for exemplar-based image denoising and representation , 2005 .

[76]  Olga Veksler,et al.  A Variable Window Approach to Early Vision , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[77]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[78]  Stephen M. Smith,et al.  SUSAN—A New Approach to Low Level Image Processing , 1997, International Journal of Computer Vision.

[79]  Dorin Comaniciu,et al.  The Variable Bandwidth Mean Shift and Data-Driven Scale Selection , 2001, ICCV.

[80]  Bruce Fischl,et al.  Adaptive Nonlocal Filtering: A Fast Alternative to Anisotropic Diffusion for Image Enhancement , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[81]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[82]  Jean-Marc Odobez,et al.  Determination of singular points in 2D deformable flow fields , 1995, Proceedings., International Conference on Image Processing.

[83]  Donald Geman,et al.  Boundary Detection by Constrained Optimization , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[84]  Charles Kervrann,et al.  A Markov random field model-based approach to unsupervised texture segmentation using local and global spatial statistics , 1995, IEEE Trans. Image Process..

[85]  Mark Nitzberg,et al.  Nonlinear Image Filtering with Edge and Corner Enhancement , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[86]  Joachim Weickert,et al.  Anisotropic diffusion in image processing , 1996 .

[87]  Danny Barash,et al.  A Fundamental Relationship between Bilateral Filtering, Adaptive Smoothing, and the Nonlinear Diffusion Equation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[88]  Suyash P. Awate,et al.  Higher-order image statistics for unsupervised, information-theoretic, adaptive, image filtering , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[89]  Tony Lindeberg,et al.  Edge Detection and Ridge Detection with Automatic Scale Selection , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[90]  Nikos Paragios,et al.  Variable Bandwidth Image Denoising Using Image-based Noise Models , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[91]  A. Goldenshluger.  On Spatial Adaptive Estimation of Nonparametric Regression , 2004 .

[92]  P. Lions,et al.  Image selective smoothing and edge detection by nonlinear diffusion. II , 1992 .

[93]  Jaakko Astola,et al.  Adaptive window size image denoising based on ICI rule , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[94]  Ljubisa Stankovic,et al.  Performance analysis of the adaptive algorithm for bias-to-variance tradeoff , 2004, IEEE Transactions on Signal Processing.

[95]  O. Lepskii.  Asymptotically Minimax Adaptive Estimation. I: Upper Bounds. Optimally Adaptive Estimates , 1992 .

[96]  Chia-Ling Tsai,et al.  The dual-bootstrap iterative closest point algorithm with application to retinal image registration , 2003, IEEE Transactions on Medical Imaging.

[97]  P. Mrázek,et al.  ON ROBUST ESTIMATION AND SMOOTHING WITH SPATIAL AND TONAL KERNELS , 2006 .