Discrimination-Based Criteria for the Evaluation of Classifiers

Evaluating the performance of classifiers is a difficult task in machine learning. Many criteria have been proposed and used in such a process. Each criterion measures some facets of classifiers. However, none is good enough for all cases. In this communication, we justify the use of discrimination measures for evaluating classifiers. The justification is mainly based on a hierarchical model for discrimination measures, which was introduced and used in the induction of decision trees.

[1]  C. Marsala,et al.  Apprentissage inductif en presence de donnees imprecises : construction et utilisation d'arbres de decision flous , 1998 .

[2]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[3]  C. Marsala,et al.  Analytical structure of hierarchical discrimination , 1999, FUZZ-IEEE'99. 1999 IEEE International Fuzzy Systems. Conference Proceedings (Cat. No.99CH36315).

[4]  Arthur Ramer,et al.  Hierarchical Model for Discrimination Measures , 1999 .

[5]  Tom Fawcett,et al.  Analysis and Visualization of Classifier Performance: Comparison under Imprecise Class and Cost Distributions , 1997, KDD.

[6]  Kevin B. Korb,et al.  The Evaluation of Predictive Learners: Some Theoretical and Empirical Results , 2001, ECML.

[7]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[8]  Thanh Ha Dang,et al.  Measures of information for inductive learning , 2004 .

[9]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[10]  Isabelle Alvarez,et al.  Explaining the Result of a Decision Tree to the End-User , 2004, ECAI.

[11]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[12]  Thorsten Joachims,et al.  KDD-Cup 2004: results and analysis , 2004, SKDD.

[13]  Ron Kohavi,et al.  The Case against Accuracy Estimation for Comparing Induction Algorithms , 1998, ICML.

[14]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Researchers , 2007 .

[15]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[16]  I. Bratko,et al.  Information-based evaluation criterion for classifier's performance , 2004, Machine Learning.