Complexity-based induction

A central problem in inductive logic programming is theory evaluation. Without some sort of preference criterion, any two theories that explain a set of examples are equally acceptable. This paper presents a scheme for evaluating alternative inductive theories based on an objective preference criterion. It strives to extract maximal redundancy from examples, transforming structure into randomness. A major strength of the method is its application to learning problems where negative examples of concepts are scarce or unavailable. A new measure calledmodel complexity is introduced, and its use is illustrated and compared with aproof complexity measure on relational learning tasks. The complementarity of model and proof complexity parallels that of model and proof-theoretic semantics. Model complexity, where applicable, seems to be an appropriate measure for evaluating inductive logic theories.

[1]  J. R. Quinlan.  Learning Logical Definitions from Relations , 1990 .

[2]  Steven Minton,et al.  Quantitative Results Concerning the Utility of Explanation-based Learning , 1988, Artif. Intell..

[3]  Ming Li,et al.  Inductive Reasoning and Kolmogorov Complexity , 1992, J. Comput. Syst. Sci..

[4]  Tom M. Mitchell,et al.  Generalization as Search , 2002 .

[5]  Patrick Henry Winston,et al.  Learning structural descriptions from examples , 1970 .

[6]  Gary James Jason,et al.  The Logic of Scientific Discovery , 1988 .

[7]  Stephen MuggletonThe.  Compression, Signiicance and Accuracy , 1992 .

[8]  Brian R. Gaines,et al.  Behaviour/structure transformations under uncertainty , 1976 .

[9]  C. S. Wetherell,et al.  Probabilistic Languages: A Review and Some Open Questions , 1980, CSUR.

[10]  Ehud Shapiro,et al.  Algorithmic Program Debugging , 1983 .

[11]  John Wylie Lloyd,et al.  Foundations of Logic Programming , 1987, Symbolic Computation.

[12]  Stephen Muggleton,et al.  A Strategy for Constructing New Predicates in First-Order Logic , 1988, EWSL.

[13]  Ray J. Solomonoff,et al.  Complexity-based induction systems: Comparisons and convergence theorems , 1978, IEEE Trans. Inf. Theory.

[14]  Jerome A. Feldman,et al.  Some Decidability Results on Grammatical Inference and Complexity , 1972, Inf. Control..

[15]  Ian H. Witten,et al.  Arithmetic coding for data compression , 1987, CACM.

[16]  Leon Sterling,et al.  The Art of Prolog , 1987, IEEE Expert.

[17]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[18]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[19]  Wray L. Buntine.  Generalized Subsumption and Its Applications to Induction and Redundancy , 1986, Artif. Intell..

[20]  Ashwin Srinivasan,et al.  Compression, Significance, and Accuracy , 1992, ML.