On the quality of neural net classifiers

This paper describes several concepts and metrics that may be used to assess various aspects of the quality of neural net classifiers. Each concept describes a property that may be taken into account by both designers and users of neural net classifiers when assessing their utility. Besides metrics for assessment of the correctness of classifiers we also introduce metrics that address certain aspects of the misclassifications. We show the applicability of the introduced quality concepts for selection among several neural net classifiers in the domain of thyroid disorders.

[1]  Lotfi A. Zadeh,et al.  MAKING COMPUTERS THINK LIKE PEOPLE , 1984 .

[2]  John Bibby,et al.  The Analysis of Contingency Tables , 1978 .

[3]  R. Nadkarni,et al.  The Quest for Quality in the Laboratory , 1991 .

[4]  Patrick K. Simpson,et al.  Artificial Neural Systems: Foundations, Paradigms, Applications, and Implementations , 1990 .

[5]  H. Toutenburg.  Fleiss, J. L.: Statistical Methods for Rates and Proportions. John Wiley & Sons, New York‐London‐Sydney‐Toronto 1973. XIII, 233 S. , 1974 .

[6]  B. Everitt,et al.  Large sample standard errors of kappa and weighted kappa. , 1969 .

[7]  S. Haykin,et al.  Neural network detection of small moving radar targets in an ocean environment , 1992, Neural Networks for Signal Processing II Proceedings of the 1992 IEEE Workshop.

[8]  Jytte Brender,et al.  Meta-Knowledge as a Means for Quality Management in Knowledge-Based Systems , 1990 .

[9]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[10]  I. Bojesen,et al.  Evaluation of the chemical parameters of four systems of radioimmunological assay. , 1986, Scandinavian journal of clinical and laboratory investigation.

[11]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[12]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Stephen José Hanson,et al.  What connectionist models learn: Learning and representation in connectionist networks , 1990, Behavioral and Brain Sciences.

[14]  A Hasman,et al.  Interobserver variation in the assessment of fetal heart rate recordings. , 1993, European journal of obstetrics, gynecology, and reproductive biology.

[15]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[16]  Jude Shavlik,et al.  An Approach to Combining Explanation-based and Neural Learning Algorithms , 1989 .

[17]  C.S. Pattichis,et al.  Artificial neural nets in computer-aided macro motor unit potential classification , 1990, IEEE Engineering in Medicine and Biology Magazine.

[18]  S. Siegel,et al.  Nonparametric Statistics for the Behavioral Sciences , 2022, The SAGE Encyclopedia of Research Design.

[19]  Jeremy Wyatt,et al.  Connectionist models in medicine: an investigation of their potential , 1989, AIME.

[20]  Robert F. Harrison,et al.  A Connectionist Aid to the Early Diagnosis of Myocardial Infarction , 1991, AIME.

[21]  W. Gerhardt,et al.  Evaluation of test data from clinical studies. I. Terminology, graphic interpretation, diagnostic strategies, and selection of sample groups. II. Critical review of the concepts of efficiency, receiver operated characteristics (ROC), and likelihood ratios. , 1986, Scandinavian journal of clinical and laboratory investigation. Supplementum.

[22]  Etienne Barnard,et al.  Avoiding false local minima by proper initialization of connections , 1992, IEEE Trans. Neural Networks.

[23]  R. Light.  Measures of response agreement for qualitative data: Some generalizations and alternatives. , 1971 .

[24]  J. Fleiss.  Statistical methods for rates and proportions , 1974 .

[25]  Timothy R. C. Read,et al.  Goodness-Of-Fit Statistics for Discrete Multivariate Data , 1988 .

[26]  L. A. Zadeh,et al.  Making computers think like people [fuzzy set theory] , 1984, IEEE Spectrum.

[27]  Lars Kai Hansen,et al.  Ensemble methods for handwritten digit recognition , 1992, Neural Networks for Signal Processing II Proceedings of the 1992 IEEE Workshop.

[28]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[29]  J. Michaelis,et al.  Evaluation of ECG Interpretation Results Obtained by Computer and Cardiologists , 1990, Methods of Information in Medicine.

[30]  Pekka Korhonen,et al.  A pareto race , 1988 .