Learning Function-Free Horn Expressions

The problem of learning universally quantified function free first order Horn expressions is studied. Several models of learning from equivalence and membership queries are considered, including the model where interpretations are examples (Learning from Interpretations), the model where clauses are examples (Learning from Entailment), models where extensional or intentional background knowledge is given to the learner (as done in Inductive Logic Programming), and the model where the reasoning performance of the learner rather than identification is of interest (Learning to Reason). We present learning algorithms for all these tasks for the class of universally quantified function free Horn expressions. The algorithms are polynomial in the number of predicate symbols in the language and the number of clauses in the target Horn expression but exponential in the arity of predicates and the number of universally quantified variables. We also provide lower bounds for these tasks by way of characterising the VC-dimension of this class of expressions. The exponential dependence on the number of variables is the main gap between the lower and upper bounds.

[1]  Wolfgang Maass,et al.  Lower bound methods and separation results for on-line learning models , 1992, Machine Learning.

[2]  Jr. Charles David Page.  Anti-unification in constraint logics: foundations and applications to learnability in first-order logic, to speed-up learning, and to deduction , 1993 .

[3]  Alfred Horn,et al.  On sentences which are true of direct unions of algebras , 1951, Journal of Symbolic Logic.

[4]  Wolfgang Maass,et al.  Lower Bound Methods and Separation Results for On-Line Learning Models , 2004, Machine Learning.

[5]  D. Angluin.  Queries and Concept Learning , 1988 .

[6]  Fritz Wysotzki,et al.  A Logical Framework for Graph Theoretical Decision Tree Learning , 1997, ILP.

[7]  Roni Khardon,et al.  Learning to Take Actions Learning to Take Actions , 1996 .

[8]  Bill Broyles.  Notes , 1907, The Classical Review.

[9]  Prasad Tadepalli,et al.  Learning Horn Definitions with Equivalence and Membership Queries , 1997, ILP.

[10]  Shan-Hwei Nienhuys-Cheng,et al.  Foundations of Inductive Logic Programming , 1997, Lecture Notes in Computer Science.

[11]  Saso Dzeroski,et al.  PAC-learnability of determinate logic programs , 1992, COLT '92.

[12]  Alvaro del Val.  Approximate Knowledge Compilation: The First Order Case , 1996, AAAI/IAAI, Vol. 1.

[13]  Stephen Muggleton,et al.  Machine Invention of First Order Predicates by Inverting Resolution , 1988, ML.

[14]  W CohenWilliam.  Pac-learning recursive logic programs , 1995 .

[15]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[16]  Abdul Sattar,et al.  Learning from Entailment of Logic Programs with Local Variables , 1998, ALT.

[17]  William W. Cohen.  Pac-Learning Recursive Logic Programs: Efficient Algorithms , 1994, J. Artif. Intell. Res..

[18]  De Raedt,et al.  Advances in Inductive Logic Programming , 1996 .

[19]  Ehud Shapiro,et al.  Inductive Inference of Theories from Facts , 1991, Computational Logic - Essays in Honor of Alan Robinson.

[20]  Bart Selman,et al.  Knowledge compilation and theory approximation , 1996, JACM.

[21]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[22]  Dan Roth,et al.  Reasoning with Models , 1994, Artif. Intell..

[23]  Michael Frazier,et al.  Learning conjunctions of Horn clauses , 2004, Machine Learning.

[24]  T. Horvv,et al.  Learning Logic Programs with Structured Background Knowledge ( Extended Abstract ) , 1995 .

[25]  Prasad Tadepalli,et al.  Theory-guided Empirical Speedup Learning of Goal Decomposition Rules , 1996, ICML.

[26]  Luc De Raedt,et al.  An overview of the interactive concept-learner and theory revisor CLINT , 1992 .

[27]  Leslie G. Valiant,et al.  Learning Disjunction of Conjunctions , 1985, IJCAI.

[28]  David Haussler,et al.  Learning Conjunctive Concepts in Structural Domains , 1989, Machine Learning.

[29]  J. C. C. McKinsey,et al.  The decision problem for some classes of sentences without quantifiers , 1943, Journal of Symbolic Logic.

[30]  Tamás Horváth,et al.  Learning logic programs by using the product homomorphism method , 1997, COLT '97.

[31]  Ehud Shapiro,et al.  Algorithmic Program Debugging , 1983 .

[32]  F. R. A. Hopgood,et al.  Machine Intelligence 5 , 1971, The Mathematical Gazette.

[33]  Michael Frazier,et al.  Learning From Entailment: An Application to Propositional Horn Sentences , 1993, ICML.

[34]  Jack Minker.  Foundations of deductive databases and logic programming , 1988 .

[35]  J. W. Lloyd,et al.  Foundations of logic programming; (2nd extended ed.) , 1987 .

[36]  Yehoshua Sagiv,et al.  Optimizing datalog programs , 1987, Foundations of Deductive Databases and Logic Programming..

[37]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[38]  L. Pitt,et al.  On the Learnability of Disjunctive Normal Form Formulas , 1995, Machine Learning.

[39]  Luc De Raedt,et al.  First-Order jk-Clausal Theories are PAC-Learnable , 1994, Artif. Intell..

[40]  Dan Roth,et al.  Learning to reason , 1994, JACM.

[41]  Claude Sammut,et al.  LEARNING CONCEPTS BY ASKING QUESTIONS , 1998 .

[42]  Mihalis Yannakakis,et al.  On the complexity of database queries (extended abstract) , 1997, PODS.

[43]  Tamás Horváth,et al.  Learning logic programs with structured background knowledge , 2001, Artif. Intell..

[44]  Stephen Muggleton,et al.  Efficient Induction of Logic Programs , 1990, ALT.

[45]  Michael Frazier,et al.  Classic learning , 1994, COLT '94.

[46]  Chen C. Chang,et al.  Model Theory: Third Edition (Dover Books On Mathematics) By C.C. Chang;H. Jerome Keisler;Mathematics , 1966 .

[47]  Michael Frazier,et al.  CLASSIC Learning , 1994, COLT.

[48]  John Wylie Lloyd,et al.  Foundations of Logic Programming , 1987, Symbolic Computation.

[49]  Luc De Raedt,et al.  Logical Settings for Concept-Learning , 1997, Artif. Intell..

[50]  Mihalis Yannakakis,et al.  On the Complexity of Database Queries , 1999, J. Comput. Syst. Sci..

[51]  Roni Khardon,et al.  Learning Range Restricted Horn Expressions , 1999, EuroCOLT.

[52]  Hiroki Arimura,et al.  Learning Acyclic First-Order Horn Sentences from Entailment , 1997, ALT.

[53]  Prasad Tadepalli,et al.  Learning First-Order Acyclic Horn Programs from Entailment , 1998, ILP.

[54]  William W. Cohen.  Pac-learning Recursive Logic Programs: Negative Results , 1994, J. Artif. Intell. Res..