On the Effects of Dimension in Discriminant Analysis

Given fixed numbers of labeled objects on which training data can be obtained, how many variables should be used for a particular discriminant algorithm? This, of course, cannot be answeredin general since it depends on the characteristics of the populations, the sample sizes, and the algorithm. Some insight is gained in this article by studying Gaussian populations and five algorithms: linear discrimination with urlknown means and known covariance, linear discrimination with unknown means and unknown covariances, quadratic discrimination with unknown covariances and two nonparametric Bayes-type algorithms having density estimates using different, kernels (Gaussian and Cauchy).

[1]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[2]  S. John.  Errors in Discrimination , 1961 .

[3]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[4]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[5]  O. J. Dunn,et al.  Probabilities of Correct Classification in Discriminant Analysis , 1966 .

[6]  D F Specht,et al.  Vectorcardiographic diagnosis using the polynomial discriminant method of pattern recognition. , 1967, IEEE transactions on bio-medical engineering.

[7]  M. R. Mickey,et al.  Estimation of Error Rates in Discriminant Analysis , 1968 .

[8]  G. F. Hughes,et al.  On the mean accuracy of statistical pattern recognizers , 1968, IEEE Trans. Inf. Theory.

[9]  P. Lachenbruch.  On Expected Probabilities of Misclassification in Discriminant Analysis, Necessary Sample Size, and a Relation with the Multiple Correlation Coefficient , 1968 .

[10]  B. Chandrasekaran,et al.  Comments on "On the mean accuracy of statistical pattern recognizers" by Hughes, G. F , 1969, IEEE Trans. Inf. Theory.

[11]  Edwin H. Chen,et al.  A Random Normal Number Generator for 32-Bit-Word Computers , 1971 .

[12]  O. J. Dunn.  Some Expected Values for Probabilities of Correct Classification in Discriminant Analysis , 1971 .

[13]  M. Sorum,et al.  Estimating the Expected and the Optimal Probabilities of Misclassification , 1972 .

[14]  Donald H. Foley.  Considerations of sample and feature size , 1972, IEEE Trans. Inf. Theory.

[15]  J. Michaelis.  SIMULATION EXPERIMENTS WITH MULTIPLE GROUP LINEAR AND QUADRATIC DISCRIMINANT ANALYSIS , 1973 .