Tractable Induction and Classification in First Order Logic Via Stochastic Matching

Learning in first-order logic (FOL) languages suffers from a specific difficulty: both induction and classification are potentially exponential in the size of hypotheses. This difficulty is usually dealt with by limiting the size of hypotheses, via either syntactic restrictions or search strategies. This paper is concerned with polynomial induction and use of FOL hypotheses with no size restrictions. This is done via stochastic matching: instead of exhaustively exploring the set of matchings between any example and any short candidate hypothesis, one stochastically explores the set of matchings between any example and any candidate hypothesis. The user sets the number of matching samples to consider and thereby controls the cost of induction and classification. One advantage of this heuristic is to allow for resource-bounded learning, without any a priori knowledge about the problem domain. Experiments on a real-world problem pertaining to organic chemistry fully demonstrate the potentialities of the approach regarding both predictive accuracy and computational cost.

[1]  Pedro M. Domingos.  Rule Induction and Instance-Based Learning: A Unified Approach , 1995, IJCAI.

[2]  Geoffrey I. Webb.  Further Experimental Evidence against the Utility of Occam's Razor , 1996, J. Artif. Intell. Res..

[3]  Ryszard S. Michalski,et al.  A theory and methodology of inductive learning , 1993 .

[4]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[5]  Michèle Sebag,et al.  Delaying the Choice of Bias: A Disjunctive Version Space Approach , 1996, ICML.

[6]  Stephen Muggleton,et al.  Bayesian inductive logic programming , 1994, COLT '94.

[7]  Dietrich Wettschereck,et al.  Relational Instance-Based Learning , 1996, ICML.

[8]  Gilles Bisson,et al.  Learning in FOL with a Similarity Measure , 1992, AAAI.

[9]  Saso Dzeroski,et al.  Inductive Logic Programming: Techniques and Applications , 1993 .

[10]  David Haussler,et al.  Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework , 1988, Artif. Intell..

[11]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[12]  Filippo Neri,et al.  Search-Intensive Concept Induction , 1995, Evolutionary Computation.

[13]  Michle Sebag,et al.  Constraint Inductive Logic Programming , 1996 .

[14]  Jean-Gabriel Ganascia,et al.  Representation Changes for Efficient Learning in Structural Domains , 1996, ICML.

[15]  Tom M. Mitchell,et al.  Generalization as Search , 2002 .

[16]  Francesco Bergadano,et al.  Guiding induction with domain theories , 1990 .