A Maximum Uncertainty LDA-Based Approach for Limited Sample Size Problems : With Application to Face Recognition

A critical issue of applying Linear Discriminant Analysis (LDA) is both the singularity and instability of the within-class scatter matrix. In practice, particularly in image recognition applications such as face recognition, there are often a large number of pixels or pre-processed features available, but the total number of training patterns is limited and commonly less than the dimension of the feature space. In this paper, a maximum uncertainty LDA-based method is proposed. It is based on a straightforward stabilisation approach for the within-class scatter matrix. In order to evaluate its effectiveness, experiments on face recognition using the well-known ORL and FERET face databases were carried out and compared with other LDA-based methods. The results indicate that our method im-proves the LDA classification performance when the within-class scatter matrix is not only singular but also poorly estimated, with or without a Principal Component Analysis intermediate step and using less linear discriminant features.

[1]  John Van Ness,et al.  The Use of Shrinkage Estimators in Linear Discriminant Analysis , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics , 2019, Wiley Series in Probability and Statistics.

[3]  Saldju Tadjudin,et al.  CLASSIFICATION OF HIGH DIMENSIONAL DATA WITH LIMITED TRAINING SAMPLES , 1998 .

[4]  William S. Rayens A role for covariance stabilization in the construction of the classical mixture surface , 1990 .

[5]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[6]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Carlos E. Thomaz,et al.  A new covariance estimate for Bayesian classifiers in biometric recognition , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Harry Wechsler,et al.  The FERET database and evaluation procedure for face-recognition algorithms , 1998, Image Vis. Comput..

[9]  Carlos E. Thomaz,et al.  A New Fisher-Based Method Applied to Face Recognition , 2003, CAIP.

[10]  SamalAshok,et al.  Automatic recognition and analysis of human faces and facial expressions , 1992 .

[11]  Jiri Matas,et al.  Effective Implementation of Linear Discriminant Analysis for Face Recognition and Verification , 1999, CAIP.

[12]  Anil K. Jain,et al.  39 Dimensionality and sample size considerations in pattern recognition practice , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[13]  Pasquale J. Dipillo Biased discriminant analysis: Evaluation of the optimum probability of misclassification , 1979 .

[14]  Jingyu Yang,et al.  Optimal FLD algorithm for facial feature extraction , 2001, SPIE Optics East.

[15]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[16]  W. Rayens,et al.  Covariance pooling and stabilization for classification , 1991 .

[17]  Ashok Samal,et al.  Automatic recognition and analysis of human faces and facial expressions: a survey , 1992, Pattern Recognit..

[18]  Di Pillo,et al.  Biased discriminant analysis : evaluation of the optimum probability of misclassification , 1979 .

[19]  Rama Chellappa,et al.  Discriminant analysis of principal components for face recognition , 1998 .

[20]  Wolfgang Härdle,et al.  Applied Multivariate Statistical Analysis: third edition , 2006 .

[21]  Jian Yang,et al.  Why can LDA be performed in PCA transformed space? , 2003, Pattern Recognit..

[22]  Jian Yang,et al.  Feature fusion: parallel strategy vs. serial strategy , 2003, Pattern Recognit..

[23]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[24]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[25]  Hua Yu,et al.  A direct LDA algorithm for high-dimensional data - with application to face recognition , 2001, Pattern Recognit..

[26]  N. Campbell Shrunken Estimators in Discriminant and Canonical Variate Analysis , 1980 .

[27]  Juyang Weng,et al.  Using Discriminant Eigenfeatures for Image Retrieval , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Ja-Chen Lin,et al.  A new LDA-based face recognition system which can solve the small sample size problem , 1998, Pattern Recognit..

[29]  W. M. Carey,et al.  Digital spectral analysis: with applications , 1986 .

[30]  Jing-Yu Yang,et al.  Algebraic feature extraction for image recognition based on an optimal discriminant criterion , 1993, Pattern Recognit..