Nonlocal Linear Image Regularization and Supervised Segmentation

A nonlocal quadratic functional of weighted differences is examined. The weights are based on image features and represent the affinity between different pixels in the image. By prescribing different formulas for the weights, one can generalize many local and nonlocal linear denoising algorithms, including the nonlocal means filter and the bilateral filter. In this framework one can easily show that continuous iterations of the generalized filter obey certain global characteristics and converge to a constant solution. The linear operator associated with the Euler–Lagrange equation of the functional is closely related to the graph Laplacian. We can thus interpret the steepest descent for minimizing the functional as a nonlocal diffusion process. This formulation allows a convenient framework for nonlocal variational minimizations, including variational denoising, Bregman iterations, and the recently proposed inverse scale space. It is also demonstrated how the steepest descent flow can be used for segmenta...

[1]  Ron Kimmel,et al.  Images as embedding maps and minimal surfaces: movies, color, and volumetric medical images , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  John D. Lafferty,et al.  Diffusion Kernels on Graphs and Other Discrete Input Spaces , 2002, ICML.

[3]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[4]  Ann B. Lee,et al.  Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[5]  M. Maggioni,et al.  GEOMETRIC DIFFUSIONS AS A TOOL FOR HARMONIC ANALYSIS AND STRUCTURE DEFINITION OF DATA PART I: DIFFUSION MAPS , 2005 .

[6]  Gjlles Aubert,et al.  Mathematical problems in image processing , 2001 .

[7]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[8]  Piotr Indyk,et al.  Similarity Search in High Dimensions via Hashing , 1999, VLDB.

[9]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[11]  Michael Elad,et al.  On the bilateral filter and ways to improve it , 2002 .

[12]  V. Caselles,et al.  Minimizing total variation flow , 2000, Differential and Integral Equations.

[13]  Ronald R. Coifman,et al.  Diffusion-driven multiscale analysis on manifolds and graphs: top-down and bottom-up constructions , 2005, SPIE Optics + Photonics.

[14]  Mikhail Belkin,et al.  Towards a Theoretical Foundation for Laplacian-Based Manifold Methods , 2005, COLT.

[15]  S. Osher,et al.  Nonlinear inverse scale space methods , 2006 .

[16]  Guillermo Sapiro,et al.  Fast image and video denoising via nonlocal means of similar neighborhoods , 2005, IEEE Signal Processing Letters.

[17]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[18]  Pietro Perona,et al.  A Factorization Approach to Grouping , 1998, ECCV.

[19]  J. Morel,et al.  On image denoising methods , 2004 .

[20]  Guillermo Sapiro,et al.  Fast image and video colorization using chrominance blending , 2006, IEEE Transactions on Image Processing.

[21]  Rolf Adams,et al.  Seeded Region Growing , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Stanley Osher,et al.  Deblurring and Denoising of Images by Nonlocal Functionals , 2005, Multiscale Model. Simul..

[23]  S. Osher,et al.  Convergence rates of convex variational regularization , 2004 .

[24]  M. N. Özişik,et al.  Unified Analysis and Solutions of Heat and Mass Diffusion , 1984 .

[25]  Guillermo Sapiro,et al.  Filling-in by joint interpolation of vector fields and gray levels , 2001, IEEE Trans. Image Process..

[26]  E. Dubois,et al.  Digital picture processing , 1985, Proceedings of the IEEE.

[27]  Michael William Newman,et al.  The Laplacian spectrum of graphs , 2001 .

[28]  Jean-Michel Morel,et al.  Neighborhood filters and PDE’s , 2006, Numerische Mathematik.

[29]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[30]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.

[31]  Charles Kervrann,et al.  Unsupervised Patch-Based Image Regularization and Representation , 2006, ECCV.

[32]  L. Rosenhead Conduction of Heat in Solids , 1947, Nature.

[33]  Yehoshua Y. Zeevi,et al.  Estimation of optimal PDE-based denoising in the SNR sense , 2006, IEEE Transactions on Image Processing.

[35]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[36]  Marie-Pierre Jolly,et al.  Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images , 2001, ICCV.

[37]  Stephen M. Smith,et al.  SUSAN—A New Approach to Low Level Image Processing , 1997, International Journal of Computer Vision.

[38]  Yair Weiss,et al.  Segmentation using eigenvectors: a unifying view , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[39]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[40]  Leonid P. Yaroslavsky,et al.  Digital Picture Processing: An Introduction , 1985 .

[41]  Ron Kimmel,et al.  From High Energy Physics to Low Level Vision , 1997, Scale-Space.

[42]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  S. Osher,et al.  Geometric Level Set Methods in Imaging, Vision, and Graphics , 2011, Springer New York.

[44]  Harry Shum,et al.  Lazy snapping , 2004, ACM Trans. Graph..

[45]  Danny Barash,et al.  A Fundamental Relationship between Bilateral Filtering, Adaptive Smoothing, and the Nonlinear Diffusion Equation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Gareth Funka-Lea,et al.  Multi-label Image Segmentation for Medical Applications Based on Graph-Theoretic Electrical Potentials , 2004, ECCV Workshops CVAMIA and MMBIA.

[47]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[48]  Jianbo Shi,et al.  Segmentation given partial grouping constraints , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Ron Kimmel,et al.  A general framework for low level vision , 1998, IEEE Trans. Image Process..

[50]  Alan L. Yuille,et al.  Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Stanley Osher,et al.  Iterative Regularization and Nonlinear Inverse Scale Space Applied to Wavelet-Based Denoising , 2007, IEEE Transactions on Image Processing.

[52]  B. Mohar THE LAPLACIAN SPECTRUM OF GRAPHS y , 1991 .

[53]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[54]  B. Nadler,et al.  Diffusion maps, spectral clustering and reaction coordinates of dynamical systems , 2005, math/0503445.

[55]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Guy Gilboa,et al.  Nonlinear Inverse Scale Space Methods for Image Restoration , 2005, VLSM.

[57]  Ronald R. Coifman,et al.  Non-stationary analysis on datasets and applications , 2006 .

[58]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[59]  Shirley Dex,et al.  JR 旅客販売総合システム(マルス)における運用及び管理について , 1991 .

[60]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[61]  Yacov Hel-Or,et al.  Real-time pattern matching using projection kernels , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Ron Kimmel,et al.  Images as Embedded Maps and Minimal Surfaces: Movies, Color, Texture, and Volumetric Medical Images , 2000, International Journal of Computer Vision.

[63]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, ACM Trans. Graph..