A Simple, Fast, and Effictive Rule Learner

We describe SLIPPER~ a new rule learner that generates rulesets by repeatedly boosting a simple, greedy, rule-builder. Like the rulesets built by other rule learners, the ensemble of rules created by SLIPPER is compact and comprehensible. This is made possible by imposing appropriate constraints on the rule-builder, and by use of a recently-proposed generalization of Adaboost called confidence-rated boosting. In spite of its relative simplicity, SLIPPER is highly scalable, and an effective learner. Experimentally, SLIPPER scales no worse than O(n log n), where n is the number of examples, and on a set of 32 benchmark problems, SLIPPER achieves lower error rates than RIPPER 20 times, and lower error rates than C4.5rules 22 times.

[1]  William W. Cohen.  Fast Eeective Rule Induction , 1995 .

[2]  Oren Etzioni,et al.  A Redundant Covering Algorithm Applied to Text Classification , 1998 .

[3]  William W. Cohen.  Cryptographic Limitations on Learning One-Clause Logic Programs , 1993, AAAI.

[4]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[5]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[6]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[7]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[8]  Johannes Fürnkranz,et al.  Incremental Reduced Error Pruning , 1994, ICML.

[9]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[10]  Johannes Fürnkranz,et al.  Integrative Windowing , 1998, J. Artif. Intell. Res..

[11]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[12]  Michael J. Pazzani,et al.  An Investigation of Noise-Tolerant Relational Concept Learning Algorithms , 1991, ML.

[13]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[14]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1990, COLT '90.

[15]  Ian H. Witten,et al.  Generating Accurate Rule Sets Without Global Optimization , 1998, ICML.

[16]  Leslie G. Valiant,et al.  Cryptographic Limitations on Learning Boolean Formulae and Finite Automata , 1993, Machine Learning: From Theory to Applications.

[17]  J. R. Quinlan.  Learning Logical Definitions from Relations , 1990 .

[18]  Pedro M. Domingos.  Unifying Instance-Based and Rule-Based Induction , 1996, Machine Learning.

[19]  Thomas G. Dietterich,et al.  Pruning Adaptive Boosting , 1997, ICML.