Mind change complexity of learning logic programs

The present paper motivates the study of mind change complexity for learning minimal models of length-bounded logic programs. It establishes ordinal mind change complexity bounds for learnability of these classes both from positive facts and from positive and negative facts. Building on Angluin’s notion of finite thickness and Wright’s work on finite elasticity, Shinohara defined the property of bounded finite thickness to give a sufficient condition for learnability of indexed families of computable languages from positive data. This paper shows that an effective version of Shinohara’s notion of bounded finite thickness gives sufficient conditions for learnability with ordinal mind change bound, both in the context of learnability from positive data and for learnability from complete (both positive and negative) data. Let Omega be a notation for the first limit ordinal. Then, it is shown that if a language defining framework yields a uniformly decidable family of languages and has effective bounded finite thickness, then for each natural number m >0, the class of languages defined by formal systems of length <= m: • is identifiable in the limit from positive data with a mind change bound of Omega (power)m; • is identifiable in the limit from both positive and negative data with an ordinal mind change bound of Omega × m. The above sufficient conditions are employed to give an ordinal mind change bound for learnability of minimal models of various classes of length-bounded Prolog programs, including Shapiro’s linear programs, Arimura and Shinohara’s depth-bounded linearly covering programs, and Krishna Rao’s depth-bounded linearly moded programs. It is also noted that the bound for learning from positive data is tight for the example classes considered.

[1]  Andris Ambainis.  The power of procrastination in inductive inference: How it depends on used ordinal notations , 1995, EuroCOLT.

[2]  Efim B. Kinber,et al.  On a Theory of Inductive Inference , 1977, FCT.

[3]  John Case,et al.  Not-So-Nearly-Minimal-Size Program Inference , 1995, GOSLER Final Report.

[4]  G. Sacks.  Higher recursion theory , 1990 .

[5]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[6]  Takeshi Shinohara,et al.  Rich Classes Inferable from Positive Data: Length-Bounded Elementary Formal Systems , 1994, Inf. Comput..

[7]  Raymond M. Smullyan,et al.  Theory of Formal Systems. (AM-47) , 1961 .

[8]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[9]  Dana Angluin,et al.  Finding Patterns Common to a Set of Strings , 1980, J. Comput. Syst. Sci..

[10]  Carl H. Smith,et al.  On the role of procrastination for machine learning , 1992, COLT '92.

[11]  R. Smullyan.  Theory of formal systems , 1962 .

[12]  Keith Wright.  Identification of unions of languages drawn from an identifiable class , 1989, COLT '89.

[13]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[14]  Abdul Sattar,et al.  Learning from Entailment of Logic Programs with Local Variables , 1998, ALT.

[15]  Arun Sharma,et al.  Generalized notions of mind change complexity , 1997, COLT '97.

[16]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[17]  Arun Sharma,et al.  Elementary formal systems, intrinsic complexity, and procrastination , 1997, COLT '96.

[18]  G. Plotkin.  Automatic Methods of Inductive Inference , 1972 .

[19]  Hiroki Arimura,et al.  Inductive Inference of Prolog Programs with Linear Data Dependency from Positive Data , 1993 .

[20]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[21]  Luc De Raedt,et al.  First-Order jk-Clausal Theories are PAC-Learnable , 1994, Artif. Intell..

[22]  Akihiro Yamamoto,et al.  Learning Elementary Formal Systems , 1992, Theor. Comput. Sci..

[23]  David Page,et al.  Learning Constrained Atoms , 1991, ML.

[24]  Roni Khardon,et al.  Learning first order universal Horn expressions , 1998, COLT' 98.

[25]  Ehud Shapiro,et al.  Inductive Inference of Theories from Facts , 1991, Computational Logic - Essays in Honor of Alan Robinson.

[26]  William W. Cohen.  Pac-Learning Recursive Logic Programs: Efficient Algorithms , 1994, J. Artif. Intell. Res..

[27]  J. W. Lloyd,et al.  Foundations of logic programming; (2nd extended ed.) , 1987 .

[28]  Jörg-Uwe Kietz,et al.  Some Lower Bounds for the Computational Complexity of Inductive Logic Programming , 1993, ECML.

[29]  Carl H. Smith,et al.  General Inductive Inference Types Based on Linearly-Ordered Sets , 1996, STACS.

[30]  Setsuo Ohsuga,et al.  Information Modelling and Knowledge Bases , 1990 .

[31]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[32]  Francesco Bergadano,et al.  Inductive Logic Programming: From Machine Learning to Software Engineering , 1995 .

[33]  Shan-Hwei Nienhuys-Cheng,et al.  Foundations of Inductive Logic Programming , 1997, Lecture Notes in Computer Science.

[34]  Saso Dzeroski,et al.  PAC-learnability of determinate logic programs , 1992, COLT '92.

[35]  Akihiro Yamamoto.  Generalized Unification as Background Knowledge in Learning Logic Programs , 1993, ALT.

[36]  William W. Cohen.  Pac-learning Recursive Logic Programs: Negative Results , 1994, J. Artif. Intell. Res..

[37]  William W. Cohen.  Pac-Learning Non-Recursive Prolog Clauses , 1995, Artif. Intell..

[38]  Stephen Cole Kleene,et al.  On notation for ordinal numbers , 1938, Journal of Symbolic Logic.

[39]  D. C. Cooper,et al.  Theory of Recursive Functions and Effective Computability , 1969, The Mathematical Gazette.

[40]  Andris Ambainis,et al.  Ordinal Mind Change Complexity of Language Identification , 1997, Theor. Comput. Sci..

[41]  Ayumi Shinohara,et al.  Learning Elementary Formal Systems and an Application to Discovering Motifs in Proteins , 1991 .

[42]  Setsuo Arikawa,et al.  Elementary formal systems and formal languages-simple formal systems , 1970 .

[43]  M. R. K. Krishna Rao.  A Class of Prolog Programs Inferable from Positive Data , 1996, ALT.