Incremental Abductive EBL

In previous work, we described a knowledge-intensive inductive learning algorithm called abductive explanation-based learning (A-EBL) that uses background knowledge to improve the performance of a concept learner. A disadvantage of A-EBL is that it is not incremental. This article describes an alternative learning algorithm called IA-EBL that learns incrementally; IA-EBL replaces the set-cover-based learning algorithm of A-EBL with an extension of a perceptron learning algorithm. IA-EBL is in most other respects comparable to A-EBL, except that the output of the learning system can no longer be easily expressed as a logical theory. In this article, IA-EBL is described, analyzed according to Littlestone's model of mistake-bounded learnability, and finally compared experimentally to A-EBL. IA-EBL is shown to provide order-of-magnitude speedups over A-EBL in two domains when used in an incremental setting.

[1]  Tom Fawcett.  Learning from Plausible Explanations , 1989, ML.

[2]  Lawrence Birnbaum,et al.  Proceedings of the eighth international workshop on Machine learning , 1991 .

[3]  J G Daugman,et al.  Information Theory and Coding , 1998 .

[4]  Pat Langley,et al.  Hill-Climbing Theories of Learning , 1987 .

[5]  R. Edelman,et al.  Portal vein thrombosis: a review. , 1992, The American journal of medicine.

[6]  Andrea Danyluk,et al.  Finding New Rules for Incomplete Theories: Induction with Explicit Biases in Varying Contexts , 1989 .

[7]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[8]  George Drastal,et al.  Induction in an Abstraction Space: A Form of Constructive Induction , 1989, IJCAI.

[9]  Avrim Blum.  Separating PAC and mistake-bound learning models over the Boolean domain (abstract) , 1990, COLT '90.

[10]  Michael J. Pazzani,et al.  A Knowledge-intensive Approach to Learning Relational Concepts , 1991, ML.

[11]  Nick Littlestone,et al.  From on-line to batch learning , 1989, COLT '89.

[12]  Jude Shavlik,et al.  Refinement ofApproximate Domain Theories by Knowledge-Based Neural Networks , 1990, AAAI.

[13]  Lisa Hellerstein,et al.  Learning in the presence of finitely or infinitely many irrelevant attributes , 1991, COLT '91.

[14]  William W. Cohen.  Learning from Textbook Knowledge: A Case Study , 1990, AAAI.

[15]  Sridhar Mahadevan.  Using Determinations in EBL: A Solution to the incomplete Theory Problem , 1989, ML.

[16]  Brian C. Williams,et al.  Diagnosing Multiple Faults , 1987, Artif. Intell..

[17]  Haym Hirsh.  Combining Empirical and Analytical Learning with Version Spaces , 1989, ML.

[18]  Francesco Bergadano,et al.  Guiding induction with domain theories , 1990 .

[19]  Paul O'Rorke,et al.  Theory Formation by Abduction: Initial Results of a Case Study Based on the Chemical Revolution , 1989, ML.

[20]  Leslie G. Valiant,et al.  On the learnability of Boolean formulae , 1987, STOC.

[21]  William W. Cohen.  Abductive Explanation-Based Learning: A Solution to the Multiple Inconsistent Explanation Problem , 1992, Machine Learning.

[22]  Gerald DeJong,et al.  Active Explanation Reduction: An Approach to the Multiple Explanations Problem , 1988, ML.

[23]  Raymond Reiter,et al.  A Theory of Diagnosis from First Principles , 1986, Artif. Intell..

[24]  Thomas G. Dietterich,et al.  A Study of Explanation-Based Methods for Inductive Learning , 1989, Machine Learning.

[25]  Kamal M. Ali,et al.  Augmenting Domain Theory for Explanation-Based Generalization , 1989, ML.

[26]  N. Littlestone.  Mistake bounds and logarithmic linear-threshold learning algorithms , 1990 .

[27]  N. Littlestone.  Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[28]  Raymond J. Mooney,et al.  Induction Over the Unexplained: Using Overly-General Domain Theories to Aid Concept Learning , 2004, Machine Learning.

[29]  William W. Cohen.  Abductive explanation-based learning: A solution to the multiple inconsistent explanation problem , 1992, Machine Learning.

[30]  Michael Pazxani.  Selecting the Best Explanation for Explanation-Based Learning , 1997 .

[31]  William W. Cohen.  Compiling prior knowledge into an explicit basis , 1992, ICML 1992.

[32]  James Wogulis.  Revising Relational Domain Theories , 1991, ML.

[33]  W. Fawcett,et al.  The kinetics of electroreduction of Cr(III) at mercury in dimethylformamide solutions , 1989 .

[34]  R. Mooney,et al.  Explanation-Based Learning: An Alternative View , 1986, Machine Learning.

[35]  Raymond J. Mooney,et al.  Changing the Rules: A Comprehensive Approach to Theory Refinement , 1990, AAAI.

[36]  Paul S. Rosenbloom,et al.  Knowledge Level and Inductive Uses of Chunking (EBL) , 1990, AAAI.

[37]  William W. Cohen.  The Generality of Overgenerality , 1991, ML.

[38]  Raymond J. Mooney,et al.  First-Order Theory Revision , 1991, ML.

[39]  Paul E. Utgoff,et al.  Incremental Induction of Decision Trees , 1989, Machine Learning.

[40]  H. Hirsh.  Incremental Version-Space Merging: A General Framework for Concept Learning , 1990 .