Low-Rank Kernel-Based Semisupervised Discriminant Analysis

Semisupervised Discriminant Analysis SDA aims at dimensionality reduction with both limited labeled data and copious unlabeled data, but it may fail to discover the intrinsic geometry structure when the data manifold is highly nonlinear. The kernel trick is widely used to map the original nonlinearly separable problem to an intrinsically larger dimensionality space where the classes are linearly separable. Inspired by low-rank representation LLR, we proposed a novel kernel SDA method called low-rank kernel-based SDA LRKSDA algorithm where the LRR is used as the kernel representation. Since LRR can capture the global data structures and get the lowest rank representation in a parameter-free way, the low-rank kernel method is extremely effective and robust for kinds of data. Extensive experiments on public databases show that the proposed LRKSDA dimensionality reduction algorithm can achieve better performance than other related kernel SDA methods.

[1]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[2]  Aleix M. Martínez,et al.  Kernel Optimization in Discriminant Analysis , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Changyin Sun,et al.  Kernel Low-Rank Representation for face recognition , 2015, Neurocomputing.

[4]  Ziqiang Wang,et al.  Multiple kernel local Fisher discriminant analysis for face recognition , 2013, Signal Process..

[5]  Feng Zhao,et al.  Learning kernel parameters for kernel Fisher discriminant analysis , 2013, Pattern Recognit. Lett..

[6]  Mehryar Mohri,et al.  On Transductive Regression , 2006, NIPS.

[7]  Shinichi Nakajima,et al.  Semi-supervised local Fisher discriminant analysis for dimensionality reduction , 2009, Machine Learning.

[8]  Jiawei Han,et al.  Semi-supervised Discriminant Analysis , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[9]  Shuicheng Yan,et al.  Semi-supervised Learning by Sparse Representation , 2009, SDM.

[10]  Shuyuan Yang,et al.  Semi-supervised classification via kernel low-rank representation graph , 2014, Knowl. Based Syst..

[11]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[12]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[13]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[14]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[15]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Feiping Nie,et al.  A unified framework for semi-supervised dimensionality reduction , 2008, Pattern Recognit..

[17]  Shuicheng Yan,et al.  Latent Low-Rank Representation for subspace segmentation and feature extraction , 2011, 2011 International Conference on Computer Vision.

[18]  Yu-Chiang Frank Wang,et al.  Low-rank matrix recovery with structural incoherence for robust face recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Yue Han,et al.  Semi-supervised low-rank representation graph for pattern recognition , 2013, IET Image Process..

[20]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[21]  Kilian Q. Weinberger,et al.  Nonlinear Dimensionality Reduction by Semidefinite Programming and Kernel Matrix Factorization , 2005, AISTATS.

[22]  Xianda Zhang,et al.  Kernel-based nonlinear discriminant analysis using minimum squared errors criterion for multiclass and undersampled problems , 2010, Signal Process..

[23]  Nicolas Le Roux,et al.  Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering , 2003, NIPS.

[24]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[25]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Daoqiang Zhang,et al.  Semi-Supervised Dimensionality Reduction ∗ , 2007 .

[27]  Peter J. Cameron,et al.  Spectral graph theory , 2004 .

[28]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Dit-Yan Yeung,et al.  Semi-Supervised Discriminant Analysis using robust path-based similarity , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Yulong Wang,et al.  Graph-Regularized Low-Rank Representation for Destriping of Hyperspectral Images , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..