Not-So-Nearly-Minimal-Size Program Inference

Freivalds defined an acceptable programming system independent criterion for learning programs for functions in which the final programs were required to be both correct and “nearly” minimal size, i.e, within a computable function of being purely minimal size. Kinber showed that this parsimony requirement on final programs severely limits learning power. Nonetheless, in, for example, scientific inference, parsimony is considered highly desirable. A limcomputable function is (by definition) one computable by a procedure allowed to change its mind finitely many times about its output. Investigated is the possibility of assuaging somewhat the limitation on learning power resulting from requiring parsimonious final programs by use of criteria which require the final, correct programs to be “not-so-nearly” minimal size, e.g.; to be within a lim-computable function of actual minimal size. It is interestingly shown that some parsimony in the final program is thereby retained, yet learning power strictly increases. Also considered are lim-computable functions as above but for which notations for constructive ordinals are used to bound the number of mind changes allowed regarding the output. This is a variant of an idea introduced by Freivalds and Smith. For this ordinal complexity bounded version of lim-computability, the power of the resultant learning criteria form strict infinite hierarchies intermediate between the computable and the lim-computable cases. Many open questions are also presented.

[1]  Yves Marcoux.  Composition is almost as good as s-1-1 , 1989, [1989] Proceedings. Structure in Complexity Theory Fourth Annual Conference.

[2]  Paul Young,et al.  An introduction to the general theory of algorithms , 1978 .

[3]  James S. Royer.  A Connotational Theory of Program Structure , 1987, Lecture Notes in Computer Science.

[4]  Rusins Frievalds.  Inductive inference of minimal programs , 1990, COLT '90.

[5]  Carl H. Smith,et al.  On the role of procrastination for machine learning , 1992, COLT '92.

[6]  Stephen Cole Kleene,et al.  On notation for ordinal numbers , 1938, Journal of Symbolic Logic.

[7]  Hartley Rogers,et al.  Gödel numberings of partial recursive functions , 1958, Journal of Symbolic Logic.

[8]  Y. Ershov.  On a hierarchy of sets, II , 1968 .

[9]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[10]  John Case,et al.  Convergence to nearly minimal size grammars by vacillating learning machines , 1989, COLT '89.

[11]  N. Shapiro.  Review: E. Mark Gold, Limiting Recursion; Hilary Putnam, Trial and Error Predicates and the Solution to a Problem of Mostowski , 1971 .

[12]  D. C. Cooper,et al.  Theory of Recursive Functions and Effective Computability , 1969, The Mathematical Gazette.

[13]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[14]  R. V. Freivald.  Minimal Gödel Numbers and Their Identification in the Limit , 1975, MFCS.

[15]  Keh-Jiann Chen,et al.  Tradeoffs in machine inductive inference , 1981 .

[16]  Keh-Jiann Chen.  Tradeoffs in the Inductive Inference of Nearly Minimal Size Programs , 1982, Inf. Control..

[17]  S. Kleene.  On the Forms of the Predicates in the Theory of Constructive Ordinals (Second Paper) , 1955 .

[18]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[19]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[20]  Hilary Putnam,et al.  Trial and error predicates and the solution to a problem of Mostowski , 1965, Journal of Symbolic Logic.

[21]  Efim B. Kinber,et al.  On a Theory of Inductive Inference , 1977, FCT.

[22]  G. Sacks.  Higher recursion theory , 1990 .

[23]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[24]  Efim B. Kinber.  A Note on Limit Identification of c-minimal Indices , 1983, J. Inf. Process. Cybern..

[25]  Gregory A. Riccardi.  The Independence of Control Structures in Abstract Programming Systems , 1981, J. Comput. Syst. Sci..

[26]  Alonzo Church,et al.  Formal definitions in the theory of ordinal numbers , 1937 .