Image classification via least square semi-supervised discriminant analysis with flexible kernel regression for out-of-sample extension

Semi-supervised dimensionality reduction is an important research topic in many pattern recognition and machine learning applications. Among all the methods for semi-supervised dimensionality reduction, SDA and LapRLS/L are two popular ones. Though the two methods are actually the extensions of different supervised methods, we show in this paper that both SDA and Lap-RLS/L can be unified under a regularized least square framework. In this paper, we propose a new effective semi-supervised dimensionality reduction method for better cope with data sampled from nonlinear manifold. In addition, the proposed method can both handle the regression as well as the subspace learning problem. Theoretical analysis and extensive simulations show the effectiveness of our algorithm. The results in simulations demonstrate that our proposed algorithm can achieve great superiority compared with other existing methods.

[1]  Fei Yin,et al.  CASIA Online and Offline Chinese Handwriting Databases , 2011, 2011 International Conference on Document Analysis and Recognition.

[2]  Jian Yang,et al.  KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[4]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[5]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[6]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[7]  Zhao Zhang,et al.  Learning from local and global discriminative information for semi-supervised dimensionality reduction , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[8]  Volker Blanz,et al.  Component-Based Face Recognition with 3D Morphable Models , 2003, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[9]  Jiawei Han,et al.  Semi-supervised Discriminant Analysis , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[10]  Wenhua Wang,et al.  Classification by semi-supervised discriminative regularization , 2010, Neurocomputing.

[11]  Tommy W. S. Chow,et al.  A general soft label based Linear Discriminant Analysis for semi-supervised dimensionality reduction , 2014, Neural Networks.

[12]  Bernt Schiele,et al.  Analyzing contour and appearance based methods for object categorization , 2003, CVPR 2003.

[13]  Ivor W. Tsang,et al.  Flexible Manifold Embedding: A Framework for Semi-Supervised and Unsupervised Dimension Reduction , 2010, IEEE Transactions on Image Processing.

[14]  Zhihua Zhang,et al.  Regularized Discriminant Analysis, Ridge Regression and Beyond , 2010, J. Mach. Learn. Res..

[15]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[18]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Jieping Ye,et al.  Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems , 2005, J. Mach. Learn. Res..

[20]  Jieping Ye,et al.  Least squares linear discriminant analysis , 2007, ICML '07.

[21]  Tommy W. S. Chow,et al.  Trace ratio criterion based generalized discriminative learning for semi-supervised dimensionality reduction , 2012, Pattern Recognit..

[22]  Jieping Ye,et al.  A scalable two-stage approach for a class of dimensionality reduction techniques , 2010, KDD.

[23]  Feiping Nie,et al.  A general kernelization framework for learning algorithms based on kernel PCA , 2010, Neurocomputing.

[24]  Tommy W. S. Chow,et al.  Soft label based Linear Discriminant Analysis for image recognition and retrieval , 2014, Comput. Vis. Image Underst..

[25]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[28]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[29]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[30]  Tommy W. S. Chow,et al.  Trace Ratio Optimization-Based Semi-Supervised Nonlinear Dimensionality Reduction for Marginal Manifold Visualization , 2013, IEEE Transactions on Knowledge and Data Engineering.

[31]  Tommy W. S. Chow,et al.  Marginal semi-supervised sub-manifold projections with informative constraints for dimensionality reduction and recognition , 2012, Neural Networks.

[32]  Xuelong Li,et al.  Patch Alignment for Dimensionality Reduction , 2009, IEEE Transactions on Knowledge and Data Engineering.

[33]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Bernt Schiele,et al.  Analyzing appearance and contour based methods for object categorization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..