Lazy naive credal classifier

We propose a local (or lazy) version of the naive credal classifier. The latter is an extension of naive Bayes to imprecise probability developed to issue reliable classifications despite small amounts of data, which may then be carrying highly uncertain information about a domain. Reliability is maintained because credal classifiers can issue set-valued classifications on instances that are particularly difficult to classify. We show by extensive experiments that the local classifier outperforms the original one, both in terms of accuracy of classification and because it leads to stronger conclusions (i.e., set-valued classifications made by fewer classes). By comparing the local credal classifier with a local version of naive Bayes, we also show that the former reliably deals with instances which are difficult to classify, unlike the local naive Bayes which leads to fragile classifications.

[1]  P. van der Putten,et al.  A Bias-Variance Analysis of a Real World Learning Problem: The CoIL Challenge 2000 , 2004 .

[2]  Marco Zaffalon,et al.  Statistical inference of the naive credal classifier , 2001, ISIPTA.

[3]  Grigorios Tsoumakas,et al.  Random k -Labelsets: An Ensemble Method for Multilabel Classification , 2007, ECML.

[4]  Mong-Li Lee,et al.  SNNB: A Selective Neighborhood Based Naïve Bayes for Lazy Learning , 2002, PAKDD.

[5]  Marco Zaffalon,et al.  JNCC2: The Java Implementation Of Naive Credal Classifier 2 , 2008 .

[6]  Mauro Birattari,et al.  Lazy Learning Meets the Recursive Least Squares Algorithm , 1998, NIPS.

[7]  Marco Zaffalon,et al.  Learning Reliable Classifiers From Small or Incomplete Data Sets: The Naive Credal Classifier 2 , 2008, J. Mach. Learn. Res..

[8]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[9]  Jerome H. Friedman,et al.  On Bias, Variance, 0/1—Loss, and the Curse-of-Dimensionality , 2004, Data Mining and Knowledge Discovery.

[10]  Maarten van Someren,et al.  A Bias-Variance Analysis of a Real World Learning Problem: The CoIL Challenge 2000 , 2004, Machine Learning.

[11]  P. Walley.  Statistical Reasoning with Imprecise Probabilities , 1990 .

[12]  Bernhard Pfahringer,et al.  Locally Weighted Naive Bayes , 2002, UAI.

[13]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[14]  Usama M. Fayyad,et al.  Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning , 1993, IJCAI.

[15]  David W. Aha,et al.  A Review and Empirical Evaluation of Feature Weighting Methods for a Class of Lazy Learning Algorithms , 1997, Artificial Intelligence Review.

[16]  Ivo D Dinov,et al.  SOCR: Statistics Online Computational Resource. , 2006, Journal of statistical software.