Robust Positive semidefinite L-Isomap Ensemble

In this paper, we derive an ensemble method inspired by boosting, a novel Robust Positive semidefinite L-Isomap Ensemble (RPL-IsomapE) approach. Specifically, we first apply a constant-shifting method to yield a symmetric positive semidefinite (SPSD) matrix. For topological stability, we also employ a method for eliminating critical outlier points using the confusion rate of all the data points. Then we align individual Robust Positive semidefinite L-Isomap (RPL-Isomap) solutions in common coordinate system through high dimensional affine transformations. Finally, we combine multiple RPL-Isomap solutions by the weighted averaging procedure according to residual variance to improve the noise-robustness of our method. Our RPL-IsomapE maintains the scalability and the speed of L-Isomap. Experiments on two images data sets and a video data set confirm the promising performance of the proposed RPL-IsomapE.

[1]  Seunghak Lee,et al.  Landmark MDS ensemble , 2009, Pattern Recognit..

[2]  D. Donoho,et al.  Hessian Eigenmaps : new locally linear embedding techniques for high-dimensional data , 2003 .

[3]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[4]  Vicente Hernández,et al.  SLEPc: A scalable and flexible toolkit for the solution of eigenvalue problems , 2005, TOMS.

[5]  Daniel Thalmann,et al.  Planar arrangement of high-dimensional biomedical data sets by isomap coordinates , 2003, 16th IEEE Symposium Computer-Based Medical Systems, 2003. Proceedings..

[6]  David J. Kriegman,et al.  Visual tracking and recognition using probabilistic appearance manifolds , 2005, Comput. Vis. Image Underst..

[7]  Bernhard Schölkopf,et al.  A Local Learning Approach for Clustering , 2006, NIPS.

[8]  John Platt,et al.  FastMap, MetricMap, and Landmark MDS are all Nystrom Algorithms , 2005, AISTATS.

[9]  Petros Drineas,et al.  On the Nyström Method for Approximating a Gram Matrix for Improved Kernel-Based Learning , 2005, J. Mach. Learn. Res..

[10]  Ann B. Lee,et al.  Diffusion maps and coarse-graining: a unified framework for dimensionality reduction, graph partitioning, and data set parameterization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[12]  Ameet Talwalkar,et al.  Ensemble Nystrom Method , 2009, NIPS.

[13]  Gene H. Golub,et al.  Matrix computations , 1983 .

[14]  Ivor W. Tsang,et al.  Improved Nyström low-rank approximation and error analysis , 2008, ICML '08.

[15]  Francis Cailliez,et al.  The analytical solution of the additive constant problem , 1983 .

[16]  Inderjit S. Dhillon,et al.  Semi-supervised graph clustering: a kernel approach , 2005, ICML '05.

[17]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Matthias W. Seeger,et al.  Using the Nyström Method to Speed Up Kernel Machines , 2000, NIPS.

[19]  Hongbin Zha,et al.  Riemannian Manifold Learning , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Maoguo Gong,et al.  Fast density-weighted low-rank approximation spectral clustering , 2011, Data Mining and Knowledge Discovery.

[21]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[22]  Hongyuan Zha,et al.  Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment , 2002, ArXiv.

[23]  Michel Verleysen,et al.  Nonlinear dimensionality reduction of data manifolds with essential loops , 2005, Neurocomputing.

[24]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[25]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[26]  David A. Landgrebe,et al.  Supervised classification in high-dimensional space: geometrical, statistical, and asymptotical properties of multivariate data , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[27]  W. Torgerson.  Multidimensional scaling: I. Theory and method , 1952 .

[28]  Olli Silven,et al.  Comparison of dimensionality reduction methods for wood surface inspection , 2003, International Conference on Quality Control by Artificial Vision.

[29]  Katsuhiko Sakaue,et al.  Head pose estimation by nonlinear manifold learning , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[30]  Yuxiao Hu,et al.  Nonlinear Discriminant Analysis on Embedded Manifold , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Joshua B. Tenenbaum,et al.  Global Versus Local Methods in Nonlinear Dimensionality Reduction , 2002, NIPS.

[32]  Ameet Talwalkar,et al.  Large-scale manifold learning , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[34]  Kilian Q. Weinberger,et al.  Learning a kernel matrix for nonlinear dimensionality reduction , 2004, ICML.

[35]  Andrew W. Moore,et al.  An Investigation of Practical Approximate Nearest Neighbor Algorithms , 2004, NIPS.

[36]  Fei Wang,et al.  Label Propagation through Linear Neighborhoods , 2008, IEEE Trans. Knowl. Data Eng..

[37]  Jitendra Malik,et al.  Spectral grouping using the Nystrom method , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Mohamed-Ali Belabbas,et al.  Spectral methods in machine learning and new strategies for very large datasets , 2009, Proceedings of the National Academy of Sciences.

[39]  R. Polikar,et al.  Ensemble based systems in decision making , 2006, IEEE Circuits and Systems Magazine.

[40]  Ming-Hsuan Yang,et al.  Higher Dimensional Affine Registration and Vision Applications , 2008, ECCV.

[41]  Joshua B. Tenenbaum,et al.  The Isomap Algorithm and Topological Stability , 2002, Science.

[42]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[43]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[44]  Bernhard Schölkopf,et al.  A kernel view of the dimensionality reduction of manifolds , 2004, ICML.

[45]  Genevieve Gorrell,et al.  Generalized Hebbian Algorithm for Incremental Singular Value Decomposition in Natural Language Processing , 2006, EACL.

[46]  Heeyoul Choi,et al.  Robust kernel Isomap , 2007, Pattern Recognit..