Pac-Learning Non-Recursive Prolog Clauses

Abstract Recently there has been an increasing amount of research on learning concepts expressed in subsets of Prolog; the term inductive logic programming (ILP) has been used to describe this growing body of research. This paper seeks to expand the theoretical foundations of ILP by investigating the pac-learnability of logic programs. We focus on programs consisting of a single function-free non-recursive clause, and focus on generalizations of a language known to be pac-learnable: namely, the language of determinate function-free clauses of constant depth. We demonstrate that a number of syntactic generalizations of this language are hard to learn, but that the language can be generalized to clauses of constant locality while still allowing pac-learnability. More specifically, we first show that determinate clauses of log depth are not pac-learnable, regardless of the language used to represent hypotheses. We then investigate the effect of allowing indeterminacy in a clause, and show that clauses with k indeterminate variables are as hard to learn as DNF. We next show that a more restricted language of clauses with bounded indeterminacy is learnable using k-CNF to represent hypotheses, and that restricting the “locality” of a clause to a constant allows pac-learnability even if an arbitrary amount of indeterminacy is allowed. This last result is also shown to be a strict generalization of the previous result for determinate function-free clauses of constant depth. Finally, we present some extensions of these results to logic programs with multiple clauses.

[1]  Leslie G. Valiant,et al.  Cryptographic limitations on learning Boolean formulae and finite automata , 1994, JACM.

[2]  David Page,et al.  Learning Constrained Atoms , 1991, ML.

[3]  J. Ross Quinlan,et al.  Determinate Literals in Inductive Logic Programming , 1991, IJCAI.

[4]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[5]  Stephen Muggleton,et al.  Protein secondary structure prediction using logic-based machine learning , 1992 .

[6]  Saso Dzeroski,et al.  Background Knowledge and Declarative Bias in Inductive Concept Learning , 1992, AII.

[7]  Stephen Muggleton,et al.  Efficient Induction of Logic Programs , 1990, ALT.

[8]  William W. Cohen.  Pac-Learning Recursive Logic Programs: Efficient Algorithms , 1994, J. Artif. Intell. Res..

[9]  Ming Li,et al.  Learning Simple Concept Under Simple Distributions , 1991, SIAM J. Comput..

[10]  Michele Flammini,et al.  Learning DNF formulae under classes of probability distributions , 1992, COLT '92.

[11]  Michael Frazier,et al.  Classic learning , 1994, COLT '94.

[12]  Michael Kharitonov,et al.  Cryptographic lower bounds for learnability of Boolean functions on the uniform distribution , 1992, COLT '92.

[13]  Michael Frazier,et al.  CLASSIC Learning , 1994, COLT.

[14]  Leonard Pitt,et al.  Reductions among prediction problems: on the difficulty of predicting automata , 1988, [1988] Proceedings. Structure in Complexity Theory Third Annual Conference.

[15]  Eyal Kushilevitz,et al.  On learning visual concepts and DNF formulae , 1993, COLT '93.

[16]  ProgramsWilliam W. CohenAT.  Learnability of Restricted Logic Programs , 1993 .

[17]  William W. Cohen.  Grammatically Biased Learning: Learning Logic Programs Using an Explicit Antecedent Description Language , 1994, Artif. Intell..

[18]  C. Feng,et al.  Temporal Decision Trees: Model-based Diagnosis of Dynamic Systems On-Board , 2003, J. Artif. Intell. Res..

[19]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[20]  Leslie G. Valiant,et al.  Computational limitations on learning from examples , 1988, JACM.

[21]  Jörg-Uwe Kietz,et al.  Some Lower Bounds for the Computational Complexity of Inductive Logic Programming , 1993, ECML.

[22]  J. Van Leeuwen,et al.  Handbook of theoretical computer science - Part A: Algorithms and complexity; Part B: Formal models and semantics , 1990 .

[23]  Ravi B. Boppana,et al.  The Complexity of Finite Functions , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[24]  Ehud Shapiro,et al.  Algorithmic Program Debugging , 1983 .

[25]  BiasWilliam W. CohenAT,et al.  Rapid Prototyping of ILP Systems Using Explicit Bias , 1993 .

[26]  Richard A. Lewis,et al.  Drug design by machine learning: the use of inductive logic programming to model the structure-activity relationships of trimethoprim analogues binding to dihydrofolate reductase. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[27]  William W. Cohen.  Pac-learning Recursive Logic Programs: Negative Results , 1994, J. Artif. Intell. Res..

[28]  Alfred V. Aho,et al.  The Design and Analysis of Computer Algorithms , 1974 .

[29]  William W. Cohen,et al.  Learnability of description logics , 1992, COLT '92.

[30]  Michael Frazier,et al.  Learning From Entailment: An Application to Propositional Horn Sentences , 1993, ICML.

[31]  Leslie G. Valiant,et al.  On the learnability of Boolean formulae , 1987, STOC.

[32]  J. Lloyd.  Foundations of Logic Programming , 1984, Symbolic Computation.

[33]  William W. Cohen.  Pac-Learning Nondeterminate Clauses , 1994, AAAI.

[34]  Thomas R. Hancock.  Learning kμ decision trees on the uniform distribution , 1993, COLT '93.

[35]  Saso Dzeroski,et al.  Inductive logic programming and learnability , 1994, SGAR.

[36]  William W. Cohen.  Cryptographic Limitations on Learning One-Clause Logic Programs , 1993, AAAI.

[37]  Melissa P. Chase,et al.  On Analytical and Similarity-Based Classification , 1990, AAAI.

[38]  Saso Dzeroski,et al.  PAC-learnability of determinate logic programs , 1992, COLT '92.

[39]  Leonard Pitt,et al.  Prediction-Preserving Reducibility , 1990, J. Comput. Syst. Sci..

[40]  Mona Singh,et al.  Learning functions of k terms , 1990, COLT '90.