Program Size Restrictions in Computational Learning

Abstract A model for a subject S learning its environment E could be described thus: S , placed in E , receives data about E , and simultaneously conjectures a sequence of hypotheses. S is said to learn E just in case the sequence of hypotheses conjectured by S stabilizes to a final hypothesis which correctly represents E . Computational learning theory provides a framework for studying problems of this nature when the subject is a machine. A natural abstraction for the notion of hypothesis is a computer program. The present paper, in the above framework of learning, presents arguments for the final hypothesis to be succinct , and introduces a plethora of formulations of such succinctness. A revelation of this study is that some of the “natural” notions of succinctness may be uninteresting because learning capability of machines under these seemingly natural constraints is dependent on the choice of programming system used to interpret hypotheses.

[1]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[2]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[3]  W. B. Roantree OCCAM'S RAZOR , 1960 .

[4]  Keh-Jiann Chen,et al.  Tradeoffs in machine inductive inference , 1981 .

[5]  Keh-Jiann Chen Tradeoffs in the Inductive Inference of Nearly Minimal Size Programs , 1982, Inf. Control..

[6]  S. Pinker Formal models of language learning , 1979, Cognition.

[7]  R. V. Freivald Minimal Gödel Numbers and Their Identification in the Limit , 1975, MFCS.

[8]  Efim B. Kinber A Note on Limit Identification of c-minimal Indices , 1983, J. Inf. Process. Cybern..

[9]  K. Wexler On extensional learnability , 1982, Cognition.

[10]  Paul Young,et al.  An introduction to the general theory of algorithms , 1978 .

[11]  Gregory A. Riccardi The Independence of Control Structures in Abstract Programming Systems , 1981, J. Comput. Syst. Sci..

[12]  D. Terence Langendoen,et al.  The vastness of natural languages , 1984 .

[13]  Kenneth Wexler,et al.  Formal Principles of Language Acquisition , 1980 .

[14]  Carl H. Smith,et al.  Inductive Inference: Theory and Methods , 1983, CSUR.

[15]  Rolf Wiehagen,et al.  Research in the theory of inductive inference by GDR mathematicians - A survey , 1980, Inf. Sci..

[16]  Ernest Addison Moody,et al.  The logic of William of Ockham , 1935 .

[17]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[18]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[19]  Hilary Putnam,et al.  Reductionism and the nature of psychology , 1973 .

[20]  James S. Royer A Connotational Theory of Program Structure , 1987, Lecture Notes in Computer Science.

[21]  Hartley Rogers,et al.  Gödel numberings of partial recursive functions , 1958, Journal of Symbolic Logic.

[22]  Jr. Hartley Rogers Theory of Recursive Functions and Effective Computability , 1969 .

[23]  John Case,et al.  Machine Inductive Inference and Language Identification , 1982, ICALP.

[24]  W. M. Thorburn,et al.  THE MYTH OF OCCAM'S RAZOR , 1918 .

[25]  Rusins Freivalds Inductive Inference of Minimal Programs , 1990, COLT.

[26]  Efim B. Kinber,et al.  On a Theory of Inductive Inference , 1977, FCT.