Assessing Predictive Accuracy in Discriminant Analysis.

The estimation of probabilities of correct classification is a primary concern in predictive discriminant analysis. Three such probabilities are: (a) the optimal hit rate, that obtained when the classification rule is based on known parameters; (b) the actual hit rate, that obtained by applying a rule based on a particular sample to future samples; and (c) the expected actual hit rate. Methods of estimating these hit rates include formulas (in the two-group case), resubstitution, and external analyses. The methods are tentatively compared via Monte Carlo sampling from two real data sets.

[1]  Geoffrey J. McLachlan,et al.  The bias of sample based posterior probabilities , 1977 .

[2]  G J McLachlan,et al.  Confidence intervals for the conditional probability of misallocation in discriminant analysis. , 1975, Biometrics.

[3]  G. McLachlan.  An Asymptotic Unbiased Technique for Estimating the Error Rates in Discriminant Analysis , 1974 .

[4]  O. J. Dunn.  Some Expected Values for Probabilities of Correct Classification in Discriminant Analysis , 1971 .

[5]  Philippe Cattin,et al.  Estimation of the predictive power of a regression model. , 1980 .

[6]  P. Lachenbruch.  An almost unbiased method of obtaining confidence intervals for the probability of misclassification in discriminant analysis. , 1967, Biometrics.

[7]  P. Lachenbruch.  On Expected Probabilities of Misclassification in Discriminant Analysis, Necessary Sample Size, and a Relation with the Multiple Correlation Coefficient , 1968 .

[8]  Carl J. Huberty,et al.  Estimation in Multiple Correlation/Prediction , 1980 .

[9]  M. R. Mickey,et al.  Estimation of Error Rates in Discriminant Analysis , 1968 .

[10]  The valuation of classification rates in stepwise discriminant analyses: Classification rates in discriminant analyses , 1978 .

[11]  D. G. Morrison,et al.  Bias in Multiple Discriminant Analysis , 1965 .

[12]  G. McLachlan,et al.  Estimation of Allocation Rates in a Cluster Analysis Context , 1985 .

[13]  S. Hora,et al.  Estimation of Error Rates in Several-Population Discriminant Analysis , 1982 .

[14]  B. Efron.  Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation , 1983 .

[15]  Anil K. Jain,et al.  39 Dimensionality and sample size considerations in pattern recognition practice , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[16]  T. W. Anderson.  An Introduction to Multivariate Statistical Analysis , 1959 .

[17]  Michael W. Browne,et al.  PREDICTIVE VALIDITY OF A LINEAR REGRESSION EQUATION , 1975 .

[18]  M. Hills.  Allocation Rules and Their Error Rates , 1966 .

[19]  Ned Glick,et al.  Additive estimators for probabilities of correct classification , 1978, Pattern Recognit..