Learning with the Knowledge of an Upper Bound on Program Size

Abstract Two learning situations are considered: machine identification of programs from graphs of recursive functions (modeling inductive hypothesis formation) and machine identification of grammars from texts of recursively enumerable languages (modeling first language acquisition). Both these learning models are extended to account for situations in which a learning machine is provided additional information in the form of knowledge about an upper bound on the minimal size program (grammar) for the function (language) being identified. For a number of such extensions, it is shown that larger classes of functions (languages) can be algorithmically identified in the presence of upper bound information. Numerous interesting relationships are shown between different models of learning, number of anomalies allowed in the inferred program (grammar), and number of anomalies allowed in the upper bound information.

[1]  Leonard Pitt,et al.  A Characterization of Probabilistic Inference , 1984, FOCS.

[2]  John Case,et al.  Machine Inductive Inference and Language Identification , 1982, ICALP.

[3]  D. Terence Langendoen,et al.  The vastness of natural languages , 1984 .

[4]  John Case,et al.  Convergence to nearly minimal size grammars by vacillating learning machines , 1989, COLT '89.

[5]  D. Osherson,et al.  Learning theory and natural language , 1984, Cognition.

[6]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[7]  Daniel N. Osherson,et al.  Aggregating Inductive Expertise , 1986, Inf. Control..

[8]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[9]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[10]  Mark A. Fulk.  A study of inductive inference machines , 1986 .

[11]  R. Smullyan.  Theory of formal systems , 1962 .

[12]  D. Osherson,et al.  A note on formal learning theory , 1982, Cognition.

[13]  Paul Young,et al.  An introduction to the general theory of algorithms , 1978 .

[14]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[15]  Arun Sharma,et al.  Language Learning by a "Team" (Extended Abstract) , 1990, ICALP.

[16]  Rolf Wiehagen,et al.  Inductive Inference with Additional Information , 1979, J. Inf. Process. Cybern..

[17]  Kenneth Wexler,et al.  Formal Principles of Language Acquisition , 1980 .

[18]  S. Pinker.  Formal models of language learning , 1979, Cognition.

[19]  James S. Royer,et al.  Inductive Inference of Approximations , 1986, Inf. Control..

[20]  Arun Sharma,et al.  Learning in the Presence of Partial Explanations , 1991, Inf. Comput..

[21]  K. Wexler.  On extensional learnability , 1982, Cognition.

[22]  Daniel N. Osherson,et al.  Ideal Learning Machines , 1982, Cogn. Sci..

[23]  Hartley Rogers,et al.  Gödel numberings of partial recursive functions , 1958, Journal of Symbolic Logic.

[24]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[25]  Carl H. Smith,et al.  The Power of Pluralism for Automatic Program Synthesis , 1982, JACM.

[26]  Carl H. Smith,et al.  On the Inference of Programs Approximately Computing the Desired Function , 1986, AII.

[27]  Daniel N. Osherson,et al.  Criteria of Language Learning , 1982, Inf. Control..

[28]  Mark A. Fulk.  Prudence and Other Conditions on Formal Language Learning , 1990, Inf. Comput..

[29]  G. P. Szegö,et al.  Mathematical Systems Theory and Economics I / II , 1969 .

[30]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[31]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[32]  John Case.  The power of vacillation , 1988, COLT '88.