A greedy, adaptive approach to learning geometry of nonlinear manifolds

In this paper, we address the problem of learning the geometry of a non-linear manifold in the ambient Euclidean space into which the manifold is embedded. We propose a bottom-up approach to manifold approximation using tangent planes where the number of planes is adaptive to manifold curvature. Also, we exploit the local linearity of the manifold to subsample the manifold data before using it to learn the manifold geometry with negligible loss of approximation accuracy. In our experiments, our proposed Geometry Preserving Union-of-Affine Subspaces algorithm shows more than a 100-times decrease in the learning time when compared to state-of-the-art manifold learning algorithm, while achieving similar approximation accuracy.

[1]  Geoffrey E. Hinton,et al.  The EM algorithm for mixtures of factor analyzers , 1996 .

[2]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[3]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[4]  H. Vincent Poor,et al.  IEEE Workshop on Statistical Signal Processing, SSP 2014, Gold Coast, Australia, June 29 - July 2, 2014 , 2014, Symposium on Software Performance.

[5]  Pascal Frossard,et al.  Tangent-based manifold approximation with locally linear models , 2012, Signal Process..

[6]  Balázs Kégl,et al.  Intrinsic Dimension Estimation Using Packing Numbers , 2002, NIPS.

[7]  Gilad Lerman,et al.  Hybrid Linear Modeling via Local Best-Fit Flats , 2010, International Journal of Computer Vision.

[8]  M. Maggioni,et al.  Multi-scale geometric methods for data sets II: Geometric Multi-Resolution Analysis , 2012 .

[9]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[10]  Pascal Frossard,et al.  Linear manifold approximation based on differences of tangents , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[11]  Geoffrey E. Hinton,et al.  Modeling the manifolds of images of handwritten digits , 1997, IEEE Trans. Neural Networks.

[12]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[13]  Gilad Lerman,et al.  Median K-Flats for hybrid linear modeling with many outliers , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[14]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.