Possibilistic Methodology for the Evaluation of Classification Algorithms

In the paper we consider the problem of the statistical evaluation and comparison of different classification algorithms. For this purpose we apply the methodology of statistical tests for testing independence in the case the multinomial distribution. We propose to use two-sample tests for the comparison of different classification algorithms. In the paper we consider only the case of the supervised classification when an external ‘expert’ evaluates the correctness of classification. The results of the proposed statistical tests are interpreted using possibilistic methodology based on indices of dominance introduced by [7].

[1]  Nitin R. Patel,et al.  A Network Algorithm for Performing Fisher's Exact Test in r × c Contingency Tables , 1983 .

[2]  Piotr A. Kowalski,et al.  Complete Gradient Clustering Algorithm for Features Analysis of X-Ray Images , 2010 .

[3]  Nitin R. Patel,et al.  ALGORITHM 643: FEXACT: a FORTRAN subroutine for Fisher's exact test on unordered r×c contingency tables , 1986, TOMS.

[4]  W. Robert Stephenson,et al.  Nonparametric Statistical Methods for Complete and Censored Data , 2004, Technometrics.

[5]  John Elder,et al.  Handbook of Statistical Analysis and Data Mining Applications , 2009 .

[6]  A. Agresti,et al.  Categorical Data Analysis , 1991, International Encyclopedia of Statistical Science.

[7]  David J. Hand,et al.  Intelligent Data Analysis: An Introduction , 2005 .

[8]  Olgierd Hryniewicz,et al.  Possibilistic decisions and fuzzy statistical tests , 2006, Fuzzy Sets Syst..

[9]  James R. Schott,et al.  Principles of Multivariate Analysis: A User's Perspective , 2002 .

[10]  James K. Yarnold,et al.  The Minimum Expectation in X 2 Goodness of Fit Tests and the Accuracy of Approximations for the Null Distribution , 1970 .

[11]  E. Pietka,et al.  Information Technologies in Biomedicine , 2008 .

[12]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[13]  Didier Dubois,et al.  Ranking fuzzy numbers in the setting of possibility theory , 1983, Inf. Sci..

[14]  Olgierd Hryniewicz,et al.  Statistics with Imprecise Data , 2009, Encyclopedia of Complexity and Systems Science.

[15]  Michael R. Berthold,et al.  Intelligent Data Analysis , 2000, Springer Berlin Heidelberg.

[16]  Robert A. Meyers,et al.  Encyclopedia of Complexity and Systems Science , 2009 .

[17]  Piotr A. Kowalski,et al.  Bayes classification of imprecise information of interval type , 2011 .