On Trading Off Consistency and Coverage in Inductive Rule Learning

Evaluation metrics for rule learning typically, in one way or another, trade off consistency and coverage. In this work, we investigate this tradeoff for three different families of rule learning heuristics, all of them featuring a parameter that implements this trade-off in different guises. These heuristics are the m-estimate, the F -measure, and the Klosgen measures. The main goals of this work are to extend our understanding of these heuristics by visualizing their behavior via isometrics in coverage space, and to determine optimal parameter settings for them. Interestingly, even though the heuristics use quite different ways for implementing this trade-off, their optimal settings realize quite similar evaluation functions. Our empirical results on a large number of datasets demonstrate that, even though we do not use any form of pruning, the quality of the rules learned with these settings outperforms standard rule learning heuristics and approaches the performance of Ripper, a state-of-the-art rule learning system that uses extensive pruning and optimization phases.

[1]  Johannes Fürnkranz,et al.  ROC ‘n’ Rule Learning—Towards a Better Understanding of Covering Algorithms , 2005, Machine Learning.

[2]  Peter A. Flach,et al.  Predictive Performance of Weghted Relative Accuracy , 2000, PKDD.

[3]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[4]  N. Lavra,et al.  Predictive Performance of Weighted Relative Accuracy , 2000 .

[5]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[6]  Bojan Cestnik,et al.  Estimating Probabilities: A Crucial Task in Machine Learning , 1990, ECAI.

[7]  J. Ross Quinlan Learning First-Order Definitions of Functions , 1996, J. Artif. Intell. Res..

[8]  Matthias Thiel Separate and Conquer Framework und disjunktive Regeln , 2005 .

[9]  Peter A. Flach,et al.  Rule Evaluation Measures: A Unifying View , 1999, ILP.

[10]  FürnkranzJohannes,et al.  ROC `n' Rule LearningTowards a Better Understanding of Covering Algorithms , 2005 .

[11]  William W. Cohen Fast Effective Rule Induction , 1995, ICML.

[12]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[13]  JOHANNES FÜRNKRANZ,et al.  Separate-and-Conquer Rule Learning , 1999, Artificial Intelligence Review.

[15]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[16]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[17]  Willi Klösgen,et al.  Problems for knowledge discovery in databases and their treatment in the statistics interpreter explora , 1992, Int. J. Intell. Syst..