Expected classification error of the Fisher linear classifier with pseudo-inverse covariance matrix

The pseudo-Fisher linear classifier is considered as the ''diagonal'' Fisher linear classifier applied to the principal components corresponding to non-zero eigenvalues of the sample covariance matrix. An asymptotic formula for the expected (generalization) error of the Fisher classifier with the pseudo-inversion is derived which explains the peaking behaviour: with an increasing number of learning observations from one up to the number of features, the generalization error first decreases, and then starts to increase.

[1]  Anil K. Jain,et al.  Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[3]  Arjun K. Gupta.  On the Equivalence of two Classification Rules , 1977 .

[4]  B. Chandrasekaran,et al.  On dimensionality and sample size in statistical pattern classification , 1971, Pattern Recognit..

[5]  Anil K. Jain,et al.  39 Dimensionality and sample size considerations in pattern recognition practice , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[6]  G. McLachlan.  Discriminant Analysis and Statistical Pattern Recognition , 1992 .

[7]  Shun-ichi Amari,et al.  Four Types of Learning Curves , 1992, Neural Computation.

[8]  Di Pillo,et al.  Biased discriminant analysis : evaluation of the optimum probability of misclassification , 1979 .

[9]  R. Duin.  Small sample size generalization , 1995 .

[10]  Robert P. W. Duin,et al.  On the accuracy of statistical pattern recognizers , 1978 .

[11]  Sarunas Raudys,et al.  Evolution and generalization of a single neurone: I. Single-layer perceptron as seven statistical classifiers , 1998, Neural Networks.

[12]  T. W. Anderson,et al.  An Introduction to Multivariate Statistical Analysis , 1959 .

[13]  Pasquale J. Dipillo.  Biased discriminant analysis: Evaluation of the optimum probability of misclassification , 1979 .

[14]  F. J. Wyman,et al.  A comparison of asymptotic error rate expansions for the sample linear discriminant function , 1990, Pattern Recognit..

[15]  Robert P. W. Duin,et al.  Stabilizing classifiers for very small sample sizes , 1996, Proceedings of 13th International Conference on Pattern Recognition.