Efficient spatiotemporal grouping using the Nystrom method

Spectral graph theoretic methods have recently shown great promise for the problem of image segmentation, but due to the computational demands, applications of such methods to spatiotemporal data have been slow to appear For even a short video sequence, the set of all pairwise voxel similarities is a huge quantity of data: one second of a 256/spl times/384 sequence captured at 30 Hz entails on the order of 10/sup 13/ pairwise similarities. The contribution of this paper is a method that substantially reduces the computational requirements of grouping algorithms based on spectral partitioning, making it feasible to apply them to very large spatiotemporal grouping problems. Our approach is based on a technique for the numerical solution of eigenfunction problems known as the Nystrom method This method allows extrapolation of the complete grouping solution using only a small number of "typical" samples. In doing so, we successfully exploit the fact that there are far fewer coherent groups in an image sequence than pixels.

[1]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[2]  L. Delves,et al.  Computational methods for integral equations: Frontmatter , 1985 .

[3]  William H. Press,et al.  Numerical Recipes in C, 2nd Edition , 1992 .

[4]  L O Hall,et al.  Review of MR image segmentation techniques using pattern recognition. , 1993, Medical physics.

[5]  Chuan Yi Tang,et al.  A 2.|E|-Bit Distributed Algorithm for the Directed Euler Trail Problem , 1993, Inf. Process. Lett..

[6]  Kim L. Boyer,et al.  Quantitative measures of change based on feature organization: eigenvalues and eigenvectors , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Edward H. Adelson,et al.  A unified mixture framework for motion segmentation: incorporating spatial coherence and estimating the number of models , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Joachim M. Buhmann,et al.  Non-parametric similarity measures for unsupervised texture segmentation and image retrieval , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Yair Weiss,et al.  Smoothness in layers: Motion segmentation using nonparametric mixture estimation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Pietro Perona,et al.  A Factorization Approach to Grouping , 1998, ECCV.

[12]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[13]  Jitendra Malik,et al.  Motion segmentation and tracking using normalized cuts , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[14]  G. Micula,et al.  Numerical Treatment of the Integral Equations , 1999 .

[15]  Michael Werman,et al.  Stochastic image segmentation by typical cuts , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[16]  M. Kubovy,et al.  Breakout Session Report: Spatiotemporal Grouping , 2000 .

[17]  M Kubovy,et al.  The emergence of visual objects in space-time. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[18]  B. Schölkopf,et al.  Sparse Greedy Matrix Approximation for Machine Learning , 2000, ICML.

[19]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.

[20]  Christopher K. I. Williams,et al.  Using the Nyström Method to Speed Up Kernel Machines , 2000, NIPS.

[21]  K. Boyer,et al.  Perceptual Organization for Artificial Vision Systems , 2000 .

[22]  Inderjit S. Dhillon,et al.  Co-clustering documents and words using bipartite spectral graph partitioning , 2001, KDD '01.