Locality sensitive batch feature extraction for high-dimensional data

For feature extraction, the dimensionality of the feature space is usually much larger than the size of training set. This is known as under sample problem. At this time, local structure is more important than global structure. In this paper, locality sensitive batch feature extraction (LSBFE) is derived based on a new gradient optimization model by exploiting both local and global discriminant structure of data manifold. With the proposed LSBFE, a set of features can be extracted simultaneously. Recognition rate is improved compared with batch feature extraction (BFE), which only considers global information. It is shown that the proposed method achieves good performance for face databases, handwritten digit database, object database and DBWorld data set.

[1]  C. Wen,et al.  Trace ratio criterion for feature extraction in classification , 2014 .

[2]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[4]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[5]  Bir Bhanu,et al.  Statistical feature fusion for gait-based human recognition , 2004, CVPR 2004.

[6]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[7]  Matti Pietikäinen,et al.  Supervised Locally Linear Embedding , 2003, ICANN.

[8]  Dacheng Tao,et al.  Biased Discriminant Euclidean Embedding for Content-Based Image Retrieval , 2010, IEEE Transactions on Image Processing.

[9]  Ning Qian,et al.  On the momentum term in gradient descent learning algorithms , 1999, Neural Networks.

[10]  Terence Sim,et al.  Discriminant Subspace Analysis: A Fukunaga-Koontz Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Demetri Terzopoulos,et al.  Multilinear subspace analysis of image ensembles , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[12]  ZhangSheng,et al.  Discriminant Subspace Analysis , 2007 .

[13]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[14]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[15]  J KriegmanDavid,et al.  Eigenfaces vs. Fisherfaces , 1997 .

[16]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[17]  Piotr Indyk,et al.  Similarity Search in High Dimensions via Hashing , 1999, VLDB.

[18]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[19]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[20]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[21]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[22]  Hakan Cevikalp,et al.  Discriminative common vectors for face recognition , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Zhengguo Li,et al.  Model-Based Online Learning With Kernels , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Chengjun Liu,et al.  Enhanced Fisher linear discriminant models for face recognition , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[25]  S. Dudoit,et al.  Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data , 2002 .

[26]  Mohammed Bellalij,et al.  The Trace Ratio Optimization Problem , 2012, SIAM Rev..

[27]  David J. Fleet,et al.  Hamming Distance Metric Learning , 2012, NIPS.

[28]  Xuelong Li,et al.  General Tensor Discriminant Analysis and Gabor Features for Gait Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[30]  Yan Chen,et al.  Error tolerance based support vector machine for regression , 2011, Neurocomputing.

[31]  Josef Kittler,et al.  Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Gerasimos Potamianos,et al.  Linear discriminant analysis for speechreading , 1998, 1998 IEEE Second Workshop on Multimedia Signal Processing (Cat. No.98EX175).

[33]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[34]  Jieping Ye,et al.  An optimization criterion for generalized discriminant analysis on undersampled problems , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[36]  Raúl Rojas,et al.  Neural Networks - A Systematic Introduction , 1996 .

[37]  Hwann-Tzong Chen,et al.  Local discriminant embedding and its variants , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[38]  W. V. McCarthy,et al.  Discriminant Analysis with Singular Covariance Matrices: Methods and Applications to Spectroscopic Data , 1995 .

[39]  Matti Pietikäinen,et al.  Supervised Locally Linear Embedding Algorithm for Pattern Recognition , 2003, IbPRIA.