Review of Methods Inspired by Algebraic-Multigrid for Data and Image Analysis Applications

Algebraic Multigrid (AMG) methods were developed originally for numerically solving Partial Differential Equations (PDE), not necessarily on structured grids. In the last two decades solvers inspired by the AMG approach, were developed for non PDE problems, including data and image analysis problems, such as clustering, segmentation, quantization and others. These solvers share a common principle in that there is a crosstalk between fine and coarse representations of the problems, with flow of information in both directions, fine-to-coarse and coarse-to-fine. This paper surveys some of these problems and the AMG-inspired algorithms for their solution.

[1]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[3]  D FalgoutRobert.  An Introduction to Algebraic Multigrid , 2006 .

[4]  S. Nash,et al.  Truncated Newton-Based Multigrid Algorithm for Centroidal Voronoi Diagram Calculation , 2012 .

[5]  S. Nash.  A multigrid approach to discretized optimization problems , 2000 .

[6]  Nasser M. Nasrabadi,et al.  Image coding using vector quantization: a review , 1988, IEEE Trans. Commun..

[7]  Shai Bagon,et al.  Discrete Energy Minimization, beyond Submodularity: Applications and Approximations , 2012, ArXiv.

[8]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Sebastian Nowozin,et al.  Decision tree fields , 2011, 2011 International Conference on Computer Vision.

[10]  R.D. Falgout,et al.  An Introduction to Algebraic Multigrid Computing , 2006, Computing in Science & Engineering.

[11]  Ronen Basri,et al.  Image Segmentation by Probabilistic Bottom-Up Aggregation and Cue Integration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Yehoshua Y. Zeevi,et al.  The farthest point strategy for progressive image sampling , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 2 - Conference B: Computer Vision & Image Processing. (Cat. No.94CH3440-5).

[13]  Pedro F. Felzenszwalb,et al.  Efficient belief propagation for early vision , 2004, CVPR 2004.

[14]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[15]  Irad Yavneh,et al.  Adaptive Multiscale Redistribution for Vector Quantization , 2005, SIAM J. Sci. Comput..

[16]  Sebastian Nowozin,et al.  A Comparative Study of Modern Inference Techniques for Discrete Energy Minimization Problems , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Hans De Sterck,et al.  Multilevel Space-Time Aggregation for Bright Field Cell Microscopy Segmentation and Tracking , 2010, Int. J. Biomed. Imaging.

[18]  A. Borzì,et al.  Algebraic multigrid methods for solving generalized eigenvalue problems , 2006 .

[19]  Ronen Basri,et al.  Fast multiscale image segmentation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[20]  J. W. Ruge,et al.  4. Algebraic Multigrid , 1987 .

[21]  Ronen Basri,et al.  Texture segmentation by multiscale aggregation of filter responses and shape elements , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[22]  William L. Briggs,et al.  A multigrid tutorial , 1987 .

[23]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[24]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Wolfgang Hackbusch,et al.  Multi-grid methods and applications , 1985, Springer series in computational mathematics.

[26]  Achi Brandt,et al.  Fast multiscale clustering and manifold identification , 2006, Pattern Recognit..

[28]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Richard Szeliski,et al.  A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Ronen Basri,et al.  Segmentation and boundary detection using multiscale intensity measurements , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[31]  Wolfgang Hackbusch,et al.  On the multi-grid method applied to difference equations , 1978, Computing.

[32]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[33]  Achi Brandt,et al.  Multigrid Techniques: 1984 Guide with Applications to Fluid Dynamics, Revised Edition , 2011 .

[34]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[35]  Qiang Du,et al.  Centroidal Voronoi Tessellations: Applications and Algorithms , 1999, SIAM Rev..

[36]  Alexander M. Bronstein,et al.  Multigrid multidimensional scaling , 2006, Numer. Linear Algebra Appl..

[37]  Ronen Basri,et al.  Hierarchy and adaptivity in segmenting visual scenes , 2006, Nature.

[38]  Nikos Komodakis,et al.  Towards More Efficient and Effective LP-Based Algorithms for MRF Optimization , 2010, ECCV.

[39]  Chenglei Yang,et al.  On centroidal voronoi tessellation—energy smoothness and fast computation , 2009, TOGS.

[40]  D. Brandt,et al.  Multi-level adaptive solutions to boundary-value problems math comptr , 1977 .

[41]  P. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 1999 .

[42]  Shai Bagon,et al.  A Unified Multiscale Framework for Discrete Energy Minimization , 2012, ArXiv.

[43]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[44]  Vladimir Kolmogorov,et al.  Optimizing Binary MRFs via Extended Roof Duality , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  K. Stüben.  Algebraic multigrid (AMG): experiences and comparisons , 1983 .

[46]  Sebastian Nowozin,et al.  Variable grouping for energy minimization , 2011, CVPR 2011.

[47]  Pavel Berkhin,et al.  A Survey of Clustering Data Mining Techniques , 2006, Grouping Multidimensional Data.

[48]  Shai Bagon,et al.  A Multiscale Framework for Challenging Discrete Optimization , 2012, ArXiv.

[49]  D. Schlesinger,et al.  TRANSFORMING AN ARBITRARY MINSUM PROBLEM INTO A BINARY ONE , 2006 .

[50]  Meirav Galun,et al.  Fundamental Limitations of Spectral Clustering , 2006, NIPS.

[51]  D. F. Watson.  Computing the n-Dimensional Delaunay Tesselation with Application to Voronoi Polytopes , 1981, Comput. J..

[52]  F. Corpet.  Multiple sequence alignment with hierarchical clustering. , 1988, Nucleic acids research.

[53]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[54]  Achi Brandt,et al.  Efficient Multilevel Eigensolvers with Applications to Data Analysis Tasks , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Qiang Du,et al.  Uniform Convergence of a Nonlinear Energy-Based Multilevel Quantization Scheme , 2008, SIAM J. Numer. Anal..

[56]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[57]  R. Gray,et al.  Vector quantization , 1984, IEEE ASSP Magazine.

[58]  Qiang Du,et al.  Acceleration schemes for computing centroidal Voronoi tessellations , 2006, Numer. Linear Algebra Appl..

[59]  Irad Yavneh,et al.  A multigrid approach to the scalar quantization problem , 2005, IEEE Transactions on Information Theory.