A Possibilistic Rule-Based Classifier

Rule induction algorithms have gained a high popularity among machine learning techniques due to the “intelligibility” of their output, when compared to other “black-box” classification methods. However, they suffer from two main drawbacks when classifying test examples: i) the multiple classification problem when many rules cover an example and are associated with different classes, and ii) the choice of a default class, which concerns the non-covering case. In this paper we propose a family of Possibilistic Rule-based Classifiers (PRCs) to deal with such problems which are an extension and a modification of the Frank and Witten’ PART algorithm. The PRCs keep the same rule learning step as PART, but differ in other respects. In particular, the PRCs learn fuzzy rules instead of crisp rules, consider weighted rules at deduction time in an unordered manner instead of rule lists. They also reduce the number of examples not covered by any rule, using a fuzzy rule set with large supports. The experiments reported show that the PRCs lead to improve the accuracy of the classical PART algorithm.

[1]  Tom Fawcett.  PRIE: a system for generating rulelists to maximize ROC performance , 2008, Data Mining and Knowledge Discovery.

[2]  Peter Clark,et al.  The CN2 Induction Algorithm , 1989, Machine Learning.

[3]  Ian H. Witten,et al.  Generating Accurate Rule Sets Without Global Optimization , 1998, ICML.

[4]  Francisco Jose Moreno-Velo,et al.  FuzzyCN2: An algorithm for extracting fuzzy classification rule lists , 2010, International Conference on Fuzzy Systems.

[5]  Eyke Hüllermeier,et al.  FURIA: an algorithm for unordered fuzzy rule induction , 2009, Data Mining and Knowledge Discovery.

[6]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[7]  Didier Dubois,et al.  Representing Default Rules in Possibilistic Logic , 1992, KR.

[8]  Jacek M. Zurada,et al.  Computational intelligence methods for rule-based data understanding , 2004, Proceedings of the IEEE.

[9]  Nada Lavrac,et al.  The Multi-Purpose Incremental Learning System AQ15 and Its Testing Application to Three Medical Domains , 1986, AAAI.

[10]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[11]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[12]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[13]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[14]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[15]  Hisao Ishibuchi,et al.  Rule weight specification in fuzzy rule-based classification systems , 2005, IEEE Transactions on Fuzzy Systems.

[16]  Henrik Boström,et al.  Classifying Uncovered Examples by Rule Stretching , 2001, ILP.

[17]  Mathieu Serrurier,et al.  Coping with exceptions in multiclass ILP problems using possibilistic logic , 2005, IJCAI.

[18]  JOHANNES FÜRNKRANZ,et al.  Separate-and-Conquer Rule Learning , 1999, Artificial Intelligence Review.

[19]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[20]  Xindong Wu,et al.  Dynamic refinement of classification rules , 2002, 14th IEEE International Conference on Tools with Artificial Intelligence, 2002. (ICTAI 2002). Proceedings..