Hybrid structure for robust dimensionality reduction

In recent years, dimensionality reduction has attracted a great deal of attention in the communities of machine learning and data mining. The basic goal of dimensionality reduction is to discover the low dimensional manifold embedded in a high dimensional space. Although some existing manifold learning algorithms (ISOMAP, LE, LLE, LTSA, etc.) can capture the local structure of data manifold, they have poor performance in some recognition tasks. This is mainly because that they cannot handle well with the ''out of sample'' problem. Moreover, these algorithms are sensitive to the choice of nearest neighbors, which is crucial in classification. To address these problems, this paper proposes a Robust Dimensionality Reduction Algorithm With Local and Global Structure (RLGS) based on a novel adaptive weighting mechanism. Hybrid structure of local and global structures is studied. By using the adaptive weight, RLGS has the capacity of adaptively exploiting non-linear structure of data manifold and is robust to parameters. Experiments demonstrate that RLGS performs better on public face databases compared with other reported algorithms.

[1]  R. Vidal,et al.  Motion segmentation with missing data using PowerFactorization and GPCA , 2004, CVPR 2004.

[2]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[3]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Robert F. Cromp,et al.  Analyzing hyperspectral data with independent component analysis , 1998, Other Conferences.

[5]  Xuelong Li,et al.  Discriminant Locally Linear Embedding With High-Order Tensor Data , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  Chun Chen,et al.  Locally Discriminative Coclustering , 2012, IEEE Transactions on Knowledge and Data Engineering.

[7]  ZhangChangshui,et al.  Flexible manifold embedding , 2010 .

[8]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Xuelong Li,et al.  Effective Feature Extraction in High-Dimensional Space , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  Hyeonjoon Moon,et al.  The FERET Evaluation Methodology for Face-Recognition Algorithms , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Ivor W. Tsang,et al.  Flexible Manifold Embedding: A Framework for Semi-Supervised and Unsupervised Dimension Reduction , 2010, IEEE Transactions on Image Processing.

[12]  Zi Huang,et al.  Self-taught dimensionality reduction on the high-dimensional small-sized data , 2013, Pattern Recognit..

[13]  I. Jolliffe.  Principal Component Analysis , 2002 .

[14]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[15]  Junbin Gao,et al.  Robust multivariate L1 principal component analysis and dimensionality reduction , 2009, Neurocomputing.

[16]  Deli Zhao,et al.  Linear local tangent space alignment and application to face recognition , 2007, Neurocomputing.

[17]  Xuelong Li,et al.  Robust Alternative Minimization for Matrix Completion , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  YanShuicheng,et al.  Graph Embedding and Extensions , 2007 .

[19]  Xuelong Li,et al.  Semisupervised Dimensionality Reduction and Classification Through Virtual Label Regression , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[20]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[21]  Xuelong Li,et al.  Supervised Gaussian Process Latent Variable Model for Dimensionality Reduction , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[22]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[23]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  D. B. Graham,et al.  Characterising Virtual Eigensignatures for General Purpose Face Recognition , 1998 .

[25]  Zi Huang,et al.  Dimensionality reduction by Mixed Kernel Canonical Correlation Analysis , 2012, Pattern Recognition.

[26]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[27]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[28]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[29]  Dongwon Lee,et al.  Semi-supervised dimensionality reduction for analyzing high-dimensional data with constraints , 2012, Neurocomputing.

[30]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[31]  Shuicheng Yan,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007 .

[32]  Jacob Goldberger,et al.  Dimensionality reduction based on non-parametric mutual information , 2012, Neurocomputing.

[33]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[34]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[35]  Hongyuan Zha,et al.  Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment , 2002, ArXiv.