A note on comparing classifiers

Recently many new classifiers have been proposed, mainly based on neural network techniques. Comparisons are needed to evaluate the performance of the new methods. It is argued that a straightforward fair comparison demands automatic classifiers with no user interaction. As this conflicts with one of the main characteristics of neural networks, their flexibility, the question whether they are better or worse than traditional techniques might be undecidable.

[1]  R. Cox,et al.  Journal of the Royal Statistical Society B , 1972 .

[2]  R. Duin.  Small sample size generalization , 1995 .

[3]  Luc Devroye,et al.  Automatic Pattern Recognition: A Study of the Probability of Error , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  W. F. Schmidt.  Neural pattern classifying systems : Theory and experiments with trainable pattern classifiers , 1994 .

[5]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[6]  Terrence J. Sejnowski,et al.  SEXNET: A Neural Network Identifies Sex From Human Faces , 1990, NIPS.

[7]  Ch Chen,et al.  Pattern recognition and artificial intelligence , 1976 .

[8]  Terrence J. Sejnowski,et al.  NETtalk: a parallel network that learns to read aloud , 1988 .

[9]  Anil K. Jain,et al.  Classifier design with Parzen Windows , 1988 .

[10]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[11]  Robert P. W. Duin,et al.  On the Choice of Smoothing Parameters for Parzen Estimators of Probability Density Functions , 1976, IEEE Transactions on Computers.

[12]  Robert P. W. Duin,et al.  An experimental comparison of neural classifiers with ‘traditional’ classifiers , 1994 .

[13]  Henrik I. Christensen,et al.  Pattern Recognition in Practice IV: Multiple Paradigms, Comparative Studies and Hybrid Systems , 1994 .

[14]  B. D. Ripley,et al.  [Neural Networks: A Review from Statistical Perspective]: Comment , 1994 .

[15]  Sei-ichiro Kamata,et al.  A neural network classifier for LANDSAT image data , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[16]  Brian D. Ripley,et al.  Neural Networks and Related Methods for Classification , 1994 .

[17]  Robert Sabourin,et al.  Off-line signature verification using directional PDF and neural networks , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[18]  J. Ross Quinlan,et al.  Simplifying Decision Trees , 1987, Int. J. Man Mach. Stud..

[19]  Terrence J. Sejnowski,et al.  Learned classification of sonar targets using a massively parallel network , 1988, IEEE Trans. Acoust. Speech Signal Process..

[20]  James A. Anderson,et al.  Neurocomputing: Foundations of Research , 1988 .

[21]  PracticeLutz Prechelt,et al.  A Study of Experimental Evaluationsof Neural Network Learning Algorithms : Current Research , 1994 .

[22]  James L. McClelland.  [Neural Networks: A Review from Statistical Perspective]: Comment: Neural Networks and Cognitive Science: Motivations and Applications , 1994 .