Combining Binary Classifiers with Imprecise Probabilities

This paper proposes a simple framework to combine binary classifiers whose outputs are imprecise probabilities (or are transformed into some imprecise probabilities, e.g., by using confidence intervals). This combination comes down to solve linear programs describing constraints over events (here, subsets of classes). The number of constraints grows linearly with the number of classifiers, making the proposed framework tractable for problems involving a relatively large number of classes. After detailing the method, we provide some first experimental results illustrating the method interests.

[1]  Chih-Jen Lin,et al.  Probability Estimates for Multi-class Classification by Pairwise Coupling , 2003, J. Mach. Learn. Res..

[2]  Philippe Smets,et al.  The Transferable Belief Model , 1994, Artif. Intell..

[3]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[4]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[5]  Marco Zaffalon.  The naive credal classifier , 2002 .

[6]  Didier Dubois,et al.  Joint propagation of probability and possibility in risk analysis: Towards a formal framework , 2007, Int. J. Approx. Reason..

[7]  Marco Zaffalon.  Exact credal treatment of missing data , 2002 .

[8]  Thierry Denoeux,et al.  Refined modeling of sensor reliability in the belief function framework using contextual discounting , 2008, Inf. Fusion.

[9]  Thierry Denoeux,et al.  A k-nearest neighbor classification rule based on Dempster-Shafer theory , 1995, IEEE Trans. Syst. Man Cybern..

[10]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[11]  Isaac Levi,et al.  The Enterprise Of Knowledge , 1980 .

[12]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[13]  Chih-Jen Lin,et al.  Generalized Bradley-Terry Models and Multi-Class Probability Estimates , 2006, J. Mach. Learn. Res..

[14]  Thierry Denoeux,et al.  Pairwise classifier combination using belief functions , 2007, Pattern Recognit. Lett..

[15]  P. Walley.  Statistical Reasoning with Imprecise Probabilities , 1990 .

[16]  Thierry Denœux.  A k-Nearest Neighbor Classification Rule Based on Dempster-Shafer Theory , 2008 .

[17]  James O. Berger,et al.  An overview of robust Bayesian analysis , 1994 .

[18]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[19]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[20]  Marco E. G. V. Cattaneo,et al.  Likelihood-Based Statistical Decisions , 2005, ISIPTA.

[21]  Marco Zaffalon,et al.  Utility-Based Accuracy Measures to Empirically Evaluate Credal Classifiers , 2011 .

[22]  Gert de Cooman,et al.  A behavioural model for vague probability assessments , 2005, Fuzzy Sets Syst..

[23]  Matthias C. M. Troffaes.  Decision making under uncertainty using imprecise probabilities , 2007, Int. J. Approx. Reason..