Pac-Learning Recursive Logic Programs: Efficient Algorithms

We present algorithms that learn certain classes of function-free recursive logic programs in polynomial time from equivalence queries. In particular, we show that a single k-ary recursive constant-depth determinate clause is learnable. Two-clause programs consisting of one learnable recursive clause and one constant-depth determinate nonrecursive clause are also learnable, if an additional "basecase" oracle is assumed. These results immediately imply the pac-learnability of these classes. Although these classes of learnable recursive programs are very constrained, it is shown in a companion paper that they are maximally general, in that generalizing either class in any natural way leads to a computationally difficult learning problem. Thus, taken together with its companion paper, this paper establishes a boundary of efficient learnability for recursive logic programs.

[1]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[2]  Phillip D. Summers,et al.  A Methodology for LISP Program Construction from Examples , 1977, J. ACM.

[3]  Alan W. Biermann,et al.  The Inference of Regular LISP Programs from Examples , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[4]  Ehud Shapiro,et al.  Algorithmic Program Debugging , 1983 .

[5]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[6]  J. W. Lloyd,et al.  Foundations of logic programming; (2nd extended ed.) , 1987 .

[7]  Dana Angluin.  Equivalence queries and approximate fingerprints , 1989, COLT '89.

[8]  Stephen Muggleton,et al.  Efficient Induction of Logic Programs , 1990, ALT.

[9]  J. Ross Quinlan,et al.  Determinate Literals in Inductive Logic Programming , 1991, IJCAI.

[10]  Stephen Muggleton,et al.  Protein secondary structure prediction using logic-based machine learning , 1992 .

[11]  Saso Dzeroski,et al.  Background Knowledge and Declarative Bias in Inductive Concept Learning , 1992, AII.

[12]  Saso Dzeroski,et al.  PAC-learnability of determinate logic programs , 1992, COLT '92.

[13]  Richard A. Lewis,et al.  Drug design by machine learning: the use of inductive logic programming to model the structure-activity relationships of trimethoprim analogues binding to dihydrofolate reductase. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[14]  Haym Hirsh.  Polynomial-Time Learning with Version Spaces , 1992, AAAI.

[15]  S. Muggleton.  Inverting Implication , 1992 .

[16]  Alan M. Frisch.  Generalization and Learnability: a Study of Constrained Atoms , 1992 .

[17]  Rüdiger Wirth,et al.  Two Methods for Improving Inductive Logic Programming Systems , 1993, ECML.

[18]  Peter Idestam-Almquist,et al.  Generalization under Implication by Recursive Anti-unification , 1993, ICML.

[19]  Francesco Bergadano,et al.  An Interactive System to Learn Functional Logic Programs , 1993, IJCAI.

[20]  Michael Frazier,et al.  Learnability in Inductive Logic Programrning: Some Basic Results and Techniques , 1993, AAAI.

[21]  Luc De Raedt,et al.  Multiple Predicate Learning , 1993, IJCAI.

[22]  BiasWilliam W. CohenAT,et al.  Rapid Prototyping of ILP Systems Using Explicit Bias , 1993 .

[23]  R. Mike Cameron-Jones,et al.  FOIL: A Midterm Report , 1993, ECML.

[24]  Raymond J. Mooney,et al.  Inducing Deterministic Prolog Parsers from Treebanks: A Machine Learning Approach , 1994, AAAI.

[25]  Luc De Raedt,et al.  First-Order jk-Clausal Theories are PAC-Learnable , 1994, Artif. Intell..

[26]  Stan Matwin,et al.  Inverting Implication with Small Training Sets , 1994, ECML.

[27]  Mark Polman,et al.  Sample PAC-Learnability in Model Inference , 1994, ECML.

[28]  Ashwin Srinivasan,et al.  Mutagenesis: ILP experiments in a non-determinate biological domain , 1994 .

[29]  William W. Cohen.  Pac-Learning Nondeterminate Clauses , 1994, AAAI.

[30]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[31]  William W. Cohen.  Pac-learning Recursive Logic Programs: Negative Results , 1994, J. Artif. Intell. Res..

[32]  William W. Cohen.  Pac-Learning Non-Recursive Prolog Clauses , 1995, Artif. Intell..

[33]  Stephen Muggleton,et al.  A Learnability Model for Universal Representations and Its Application to Top-down Induction of Decision Trees , 1995, Machine Intelligence 15.