Learning Decision Rules by Randomized Iterative Local Search

Learning easily understandable decision rules from examples is one of the classic problems in machine learning. Most learning systems for this problem employ some variation of a greedy separate-and-conquer algorithm, which makes the rules order-dependent, and hence diff icult to understand. In this paper, we describe a system called LERILS that learns highly accurate and comprehensible rules from examples using a randomized iterative local search. We compare its performance to C4.5, RIPPER, CN2, G-NET, Smog, and BruteDL, and show that it compares favorably in accuracy and simplicity of hypotheses in a number of domains.

[1]  Henry Kautz,et al.  Noise Strategies for Local Search , 1994, AAAI 1994.

[2]  Sholom M. Weiss,et al.  Lightweight Rule Induction , 2000, ICML.

[3]  N. Shinkura,et al.  Pushing the envelope: chromatin boundaries at the nuclear pore. , 2002, Molecular cell.

[4]  Peter Clark,et al.  Rule Induction with CN2: Some Recent Improvements , 1991, EWSL.

[5]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[6]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[7]  Yoram Singer,et al.  A simple, fast, and effective rule learner , 1999, AAAI 1999.

[8]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[9]  Bart Selman,et al.  Pushing the Envelope: Planning, Propositional Logic and Stochastic Search , 1996, AAAI/IAAI, Vol. 2.

[10]  Alberto L. Sangiovanni-Vincentelli,et al.  Using the minimum description length principle to infer reduced ordered decision graphs , 1996, Machine Learning.

[11]  Jorma Rissanen,et al.  Universal coding, information, prediction, and estimation , 1984, IEEE Trans. Inf. Theory.

[12]  Hector J. Levesque,et al.  A New Method for Solving Hard Satisfiability Problems , 1992, AAAI.

[13]  Peter Clark,et al.  The CN2 Induction Algorithm , 1989, Machine Learning.

[14]  Sholom M. Weiss,et al.  Reduced Complexity Rule Induction , 1991, IJCAI.

[15]  Oren Etzioni,et al.  Learning Decision Lists Using Homogeneous Rules , 1994, AAAI.

[16]  金田 重郎,et al.  C4.5: Programs for Machine Learning (書評) , 1995 .

[17]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[18]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[19]  Sholom M. Weiss,et al.  Maximizing the Predictive Value of Production Rules , 1990, Artif. Intell..

[20]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .