Multi-manifold Discriminant Isomap for visualization and classification

Isomap aims to recover the intrinsic geometric structure of manifold by preserving geodesic distances between all pairs of data points. However it is an unsupervised dimensionality-reduction method. Usually, using class label information can increase the discriminating capability, hence a new supervised Isomap is proposed in this paper, dubbed Multi-manifold Discriminant Isomap (MMD-Isomap). First, data points are partitioned into different manifolds according to their class label information. Then, MMD-Isomap aims at seeking an optimal nonlinear subspace to preserve the geometrical structure of each manifold according to the Isomap criterion, meanwhile, to enhance the discriminating capability by maximizing the distances between data points of different manifolds. Finally, the corresponding optimization problem is solved by using a majorization algorithm. Furthermore, two new numerical metrics are designed to measure the performance of dimensionality-reduction method. In both visualization and classification experiments, MMD-Isomap achieves improved performance over many state-of-the-art methods. Preserve the geometrical structure and enhance the discriminating capability.SMACOF algorithm is introduced to solve the optimization problem.Two metrics are designed to evaluate dimensionality reduction methods.Experiments show the effectiveness of the method in visualization and classification.

[1]  Gang Wang,et al.  Discriminative multi-manifold analysis for face recognition from a single training sample per person , 2011, 2011 International Conference on Computer Vision.

[2]  Feiping Nie,et al.  Trace Ratio Problem Revisited , 2009, IEEE Transactions on Neural Networks.

[3]  M. Sniedovich.  Dynamic programming : foundations and principles , 2011 .

[4]  Jun Li,et al.  Nonparametric discriminant multi-manifold learning for dimensionality reduction , 2015, Neurocomputing.

[5]  Lei Zhang,et al.  A multi-manifold discriminant analysis method for image feature extraction , 2011, Pattern Recognit..

[6]  W. Wong,et al.  Supervised optimal locality preserving projection , 2012, Pattern Recognit..

[7]  Yu Zhang,et al.  Nonlinear Expectation Maximization Estimator for TDOA Localization , 2014, IEEE Wireless Communications Letters.

[8]  Hongping Cai,et al.  Learning Linear Discriminant Projections for Dimensionality Reduction of Image Descriptors , 2011, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Shuicheng Yan,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007 .

[10]  J. Leeuw,et al.  Simple and Canonical Correspondence Analysis Using the R Package anacor , 2007 .

[11]  Laurens van der Maaten,et al.  Accelerating t-SNE using tree-based algorithms , 2014, J. Mach. Learn. Res..

[12]  Patrick Mair,et al.  Multidimensional Scaling Using Majorization: SMACOF in R , 2008 .

[13]  Zi Huang,et al.  Dimensionality reduction by Mixed Kernel Canonical Correlation Analysis , 2012, Pattern Recognition.

[14]  Bo Yang,et al.  Linear dimensionality reduction based on Hybrid structure preserving projections , 2016, Neurocomputing.

[15]  Zi Huang,et al.  Self-taught dimensionality reduction on the high-dimensional small-sized data , 2013, Pattern Recognit..

[16]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[17]  Yiu-ming Cheung,et al.  Semi-Supervised Maximum Margin Clustering with Pairwise Constraints , 2012, IEEE Transactions on Knowledge and Data Engineering.

[18]  Tommy W. S. Chow,et al.  M-Isomap: Orthogonal Constrained Marginal Isomap for Nonlinear Dimensionality Reduction , 2013, IEEE Transactions on Cybernetics.

[19]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[20]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  P. Groenen,et al.  Modern multidimensional scaling , 1996 .

[22]  Hwann-Tzong Chen,et al.  Local discriminant embedding and its variants , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  Matti Pietikäinen,et al.  Supervised Locally Linear Embedding , 2003, ICANN.

[24]  Fadi Dornaika,et al.  Exponential Local Discriminant Embedding and Its Application to Face Recognition , 2013, IEEE Transactions on Cybernetics.

[25]  Jim Jing-Yan Wang,et al.  Discriminative sparse coding on multi-manifolds , 2013, Knowl. Based Syst..

[26]  W. Scott Spangler,et al.  Class visualization of high-dimensional data with applications , 2002, Comput. Stat. Data Anal..

[27]  Dong Xu,et al.  Trace Ratio vs. Ratio Trace for Dimensionality Reduction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Dimitrios Gunopulos,et al.  Non-linear dimensionality reduction techniques for classification and visualization , 2002, KDD.

[29]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[30]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[31]  Peng Li,et al.  Distance Metric Learning with Eigenvalue Optimization , 2012, J. Mach. Learn. Res..

[32]  Tommy W. S. Chow,et al.  Trace Ratio Optimization-Based Semi-Supervised Nonlinear Dimensionality Reduction for Marginal Manifold Visualization , 2013, IEEE Transactions on Knowledge and Data Engineering.

[33]  Alexander M. Bronstein,et al.  Nonlinear Dimensionality Reduction by Topologically Constrained Isometric Embedding , 2010, International Journal of Computer Vision.

[34]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[35]  Hongbin Zha,et al.  Riemannian Manifold Learning , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Xiangyang Xue,et al.  Discriminant neighborhood embedding for classification , 2006, Pattern Recognit..

[37]  Zhi-Hua Zhou,et al.  Supervised nonlinear dimensionality reduction for visualization and classification , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[38]  T. Cacoullos,et al.  Discriminant analysis and applications , 1974 .

[39]  Ameet Talwalkar,et al.  Sampling Methods for the Nyström Method , 2012, J. Mach. Learn. Res..

[40]  Xuelong Li,et al.  Discriminant Locally Linear Embedding With High-Order Tensor Data , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[41]  S. Dragomir.  A Survey on Cauchy-Buniakowsky-Schwartz Type Discrete Inequalities , 2003 .

[42]  Bo Yang,et al.  Learning discriminant isomap for dimensionality reduction , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[43]  Jiwen Lu,et al.  Multi-feature multi-manifold learning for single-sample face recognition , 2014, Neurocomputing.

[44]  Stefan Hougardy,et al.  The Floyd-Warshall algorithm on graphs with negative cycles , 2010, Inf. Process. Lett..

[45]  Jiawei Han,et al.  Document clustering using locality preserving indexing , 2005, IEEE Transactions on Knowledge and Data Engineering.

[46]  James F. Peters,et al.  Multi-manifold LLE learning in pattern recognition , 2015, Pattern Recognit..

[47]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[48]  Nicolas Le Roux,et al.  Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering , 2003, NIPS.

[49]  Nanning Zheng,et al.  Statistical Learning and Pattern Analysis for Image and Video Processing , 2009, Advances in Pattern Recognition.

[50]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.