Hierarchical Eigensolver for Transition Matrices in Spectral Methods

We show how to build hierarchical, reduced-rank representation for large stochastic matrices and use this representation to design an efficient algorithm for computing the largest eigenvalues, and the corresponding eigenvectors. In particular, the eigen problem is first solved at the coarsest level of the representation. The approximate eigen solution is then interpolated over successive levels of the hierarchy. A small number of power iterations are employed at each stage to correct the eigen solution. The typical speedups obtained by a Matlab implementation of our fast eigensolver over a standard sparse matrix eigensolver [13] are at least a factor of ten for large image sizes. The hierarchical representation has proven to be effective in a min-cut based segmentation algorithm that we proposed recently [8].

[1]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[2]  Jianbo Shi,et al.  A Random Walks View of Spectral Segmentation , 2001, AISTATS.

[3]  David Harel,et al.  ACE: a fast multiscale eigenvectors computation for drawing huge graphs , 2002, IEEE Symposium on Information Visualization, 2002. INFOVIS 2002..

[4]  Horst D. Simon,et al.  Fast multilevel implementation of recursive spectral bisection for partitioning unstructured problems , 1994, Concurr. Pract. Exp..

[5]  Chao Yang,et al.  ARPACK users' guide - solution of large-scale eigenvalue problems with implicitly restarted Arnoldi methods , 1998, Software, environments, tools.

[6]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[7]  Hongyuan Zha,et al.  Low-Rank Matrix Approximation Using the Lanczos Bidiagonalization Process with Applications , 1999, SIAM J. Sci. Comput..

[8]  Allan D. Jepson,et al.  Spectral Embedding and Min Cut for Image Segmentation , 2004, BMVC.

[9]  Ronen Basri,et al.  Fast multiscale image segmentation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[10]  Matthias W. Seeger,et al.  Using the Nyström Method to Speed Up Kernel Machines , 2000, NIPS.

[11]  A. Jepson,et al.  Spectral methods for multi-scale feature extraction and data clustering , 2004 .

[12]  Jitendra Malik,et al.  Efficient spatiotemporal grouping using the Nystrom method , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  Guy L. Scott,et al.  Feature grouping by 'relocalisation' of eigenvectors of the proximity matrix , 1990, BMVC.

[14]  László Lovász,et al.  Random Walks on Graphs: A Survey , 1993 .

[15]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Juan Lin,et al.  Reduced Rank Approximations of Transition Matrices , 2003, AISTATS.

[17]  Jitendra Malik,et al.  Spectral Partitioning with Indefinite Kernels Using the Nyström Extension , 2002, ECCV.

[18]  Alex Pothen,et al.  Graph Partitioning Algorithms with Applications to Scientific Computing , 1997 .

[19]  Dimitris Achlioptas,et al.  Fast computation of low rank matrix approximations , 2001, STOC '01.

[20]  H. Simon.  Low rank matrix approximation using the Lanczos Bidiagonalization Process , 1997 .

[21]  Alan M. Frieze,et al.  Fast Monte-Carlo algorithms for finding low-rank approximations , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[22]  Kun Huang,et al.  A unifying theorem for spectral embedding and clustering , 2003, AISTATS.

[23]  Allan D. Jepson,et al.  Half-Lives of EigenFlows for Spectral Clustering , 2002, NIPS.

[24]  Bernhard Schölkopf,et al.  Sampling Techniques for Kernel Methods , 2001, NIPS.

[25]  Geoffrey J. McLachlan,et al.  Mixture models : inference and applications to clustering , 1989 .