Learning recursive functions: A survey

Studying the learnability of classes of recursive functions has attracted considerable interest for at least four decades. Starting with Gold's (1967) model of learning in the limit, many variations, modifications and extensions have been proposed. These models differ in some of the following: the mode of convergence, the requirements intermediate hypotheses have to fulfill, the set of allowed learning strategies, the source of information available to the learner during the learning process, the set of admissible hypothesis spaces, and the learning goals. A considerable amount of work done in this field has been devoted to the characterization of function classes that can be learned in a given model, the influence of natural, intuitive postulates on the resulting learning power, the incorporation of randomness into the learning process, the complexity of learning, among others. On the occasion of Rolf Wiehagen's 60th birthday, the last four decades of research in that area are surveyed, with a special focus on Rolf Wiehagen's work, which has made him one of the most influential scientists in the theory of learning recursive functions.

[1]  Gunter Grieser.  Reflective Inductive Inference of Recursive Functions , 2002, ALT.

[2]  Boris A. Trakhtenbrot,et al.  Finite automata : behavior and synthesis , 1973 .

[3]  David Haussler,et al.  Equivalence of models for polynomial learnability , 1988, COLT '88.

[4]  T. Zeugmann,et al.  Consistent and Coherent Learning with δ-Delay , 2008 .

[5]  Manuel Blum,et al.  Inductive inference and unsolvability , 1991, Journal of Symbolic Logic.

[6]  Naoki Abe,et al.  Collaborative Filtering Using Weighted Majority Prediction Algorithms , 1998, ICML.

[7]  E. Mark Gold,et al.  Limiting recursion , 1965, Journal of Symbolic Logic.

[8]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[9]  Rolf Wiehagen.  Limes-Erkennung rekursiver Funktionen durch spezielle Strategien , 1975, J. Inf. Process. Cybern..

[10]  John Case,et al.  Robust learning--rich and poor , 2004, J. Comput. Syst. Sci..

[11]  Stuart A. Kurtz,et al.  On the role of search for learning from examples , 2001, J. Exp. Theor. Artif. Intell..

[12]  Frank Stephan,et al.  Avoiding coding tricks by hyperrobust learning , 2002, Theor. Comput. Sci..

[13]  Andris Ambainis,et al.  Probabilistic inductive inference: a survey , 1999, Theor. Comput. Sci..

[14]  Sanjay Jain,et al.  Inductive Inference , 2010, Encyclopedia of Machine Learning.

[15]  Thomas Zeugmann,et al.  On the Nonboundability of total Effective Operators , 1984, Math. Log. Q..

[16]  Thomas Zeugmann.  On Barzdin's Conjecture , 1986, AII.

[17]  Klaus P. Jantke,et al.  Natural Properties of Strategies Identifying Recursive Functions , 1979, J. Inf. Process. Cybern..

[18]  Rolf Wiehagen,et al.  Characterization Problems in the Theory of Inductive Inference , 1978, ICALP.

[19]  Leonard Pitt,et al.  Probabilistic inductive inference , 1989, JACM.

[20]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[21]  Gary James Jason,et al.  The Logic of Scientific Discovery , 1988 .

[22]  Jerome A. Feldman,et al.  Some Decidability Results on Grammatical Inference and Complexity , 1972, Inf. Control..

[23]  Carl H. Smith,et al.  Robust Learning Is Rich , 2001, J. Comput. Syst. Sci..

[24]  Rusins Freivalds,et al.  On the Power of Inductive Inference from Good Examples , 1993, Theor. Comput. Sci..

[25]  Thomas Zeugmann.  Inductive Inference of Optimal Programs: A Survey and Open Problems , 1990, Nonmonotonic and Inductive Logic.

[26]  Mark A. Fulk.  ROBUST SEPARATIONS IN INDUCTIVE INFERENCE , 1990, COLT 1990.

[27]  Albert R. Meyer,et al.  Classes of computable functions defined by bounds on computation: Preliminary Report , 1969, STOC.

[28]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[29]  B. Schölkopf,et al.  Prediction on a Graph with a Perceptron , 2007 .

[30]  Carl H. Smith,et al.  Choosing a learning team: a topological approach , 1994, STOC '94.

[31]  Carl H. Smith,et al.  On the Intrinsic Complexity of Learning , 1995, Inf. Comput..

[32]  Sandra Zilles,et al.  An approach to intrinsic complexity of uniform learning , 2006, Theor. Comput. Sci..

[33]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[34]  Carl H. Smith,et al.  Learning programs with an easy to calculate set of errors , 1988, COLT '88.

[35]  N. Littlestone.  Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[36]  Rusins Freivalds,et al.  Closedness properties in ex-identification , 2001, Theor. Comput. Sci..

[37]  Thomas Zeugmann,et al.  Incremental Learning from Positive Data , 1996, J. Comput. Syst. Sci..

[38]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part I , 1964, Inf. Control..

[39]  Robert P. Daley.  On the Error Correcting Power of Pluralism in BC-Type Inductive Inference , 1983, Theoretical Computer Science.

[40]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[41]  Rolf Wiehagen.  A Thesis in Inductive Inference , 1990, Nonmonotonic and Inductive Logic.

[42]  Sandra Zilles.  Uniform learning of recursive functions , 2003 .

[43]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[44]  Arun Sharma,et al.  Learning with the Knowledge of an Upper Bound on Program Size , 1993, Inf. Comput..

[45]  R. Richardson.  The International Congress of Mathematicians , 1932, Science.

[46]  Manfred K. Warmuth,et al.  Learning binary relations using weighted majority voting , 2004, Machine Learning.

[47]  A. Sorbi.  complexity, logic, and recursion theory , 1997 .

[48]  Andris Ambainis,et al.  Transformations that Preserve Learnability , 1996, ALT.

[49]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[50]  Carl H. Smith,et al.  A Recursive Introduction to the Theory of Computation , 1994, Graduate Texts in Computer Science.

[51]  Sandra Zilles.  On the Synthesis of Strategies Identifying Recursive Functions , 2001, COLT/EuroCOLT.

[52]  Martin Anthony,et al.  Computational Learning Theory , 1992 .

[53]  P. Young,et al.  Easy constructions in complexity theory: Gap and speed-up theorems , 1973 .

[54]  P. Odifreddi.  Classical recursion theory , 1989 .

[55]  Rolf Wiehagen,et al.  Research in the theory of inductive inference by GDR mathematicians - A survey , 1980, Inf. Sci..

[56]  Allan Borodin,et al.  Computational Complexity and the Existence of Complexity Gaps , 1972, JACM.

[57]  On the role of search for learning , 1989, COLT '89.

[58]  Carl H. Smith,et al.  Trade-Off among Parameters Effecting Inductive Inference , 1989, Inf. Comput..

[59]  Sandra Zilles.  Separation of uniform learning classes , 2004, Theor. Comput. Sci..

[60]  H. Putnam.  Mathematics, Matter, And Method , 1975 .

[61]  Jochen Nessel.  Learnability of Enumerable Classes of Recursive Functions from "Typical" Examples , 1999, ALT.

[62]  Rusins Freivalds.  Inductive Inference of Minimal Programs , 1990, COLT.

[63]  Eliana Minicozzi,et al.  Some Natural Properties of Strong-Identification in Inductive Inference , 1976, Theor. Comput. Sci..

[64]  Rusins Freivalds,et al.  On the Power of Probabilistic Strategies in Inductive Inference , 1984, Theor. Comput. Sci..

[65]  Rusins Freivalds,et al.  Probabilistic Versus Deterministic Inductive Inference in Nonstandard Numberings , 1988, Math. Log. Q..

[66]  Rusins Freivalds,et al.  Inductive Inference of Recursive Functions: Complexity Bounds , 1991, Baltic Computer Science.

[67]  Nader H. Bshouty,et al.  Asking Questions to Minimize Errors , 1996, J. Comput. Syst. Sci..

[68]  Leonard Pitt,et al.  On the Necessity of Occam Algorithms , 1992, Theor. Comput. Sci..

[69]  Carl H. Smith,et al.  On the intrinsic complexity of learning recursive functions , 2003, Inf. Comput..

[70]  Robert L. Constable.  The Operator Gap , 1969, SWAT.

[71]  Robert P. Daley.  Towards the Development of an Analysis of Learning Algorithms (Invited Key Note) , 1986, AII.

[72]  Stuart C. Shapiro,et al.  Encyclopedia of artificial intelligence, vols. 1 and 2 (2nd ed.) , 1992 .

[73]  Keh-Jiann Chen,et al.  Tradeoffs in machine inductive inference , 1981 .

[74]  Keh-Jiann Chen.  Tradeoffs in the Inductive Inference of Nearly Minimal Size Programs , 1982, Inf. Control..

[75]  Rolf Wiehagen,et al.  Charakteristische Eigenschaften von erkennbaren Klassen rekursiver Funktionen , 1976, J. Inf. Process. Cybern..

[76]  J. Helm.  On Effectively Computable Operators , 1971 .

[77]  Steffen Lange,et al.  Algorithmic Learning for Knowledge-Based Systems: Gosler Final Report , 1995 .

[78]  R. Harré,et al.  Probability and Confirmation , 1970 .

[79]  Rolf Wiehagen,et al.  Ignoring data may be the only way to learn efficiently , 1994, J. Exp. Theor. Artif. Intell..

[80]  Rolf Wiehagen,et al.  Inductive Inference with Additional Information , 1979, J. Inf. Process. Cybern..

[81]  Carl H. Smith,et al.  Inductive Inference: Theory and Methods , 1983, CSUR.

[82]  Stuart A. Kurtz,et al.  A Refutation of Barzdins' Conjecture , 1989, AII.

[83]  J. M. Barzdin,et al.  Prognostication of Automata and Functions , 1971, IFIP Congress.

[84]  Carl H. Smith,et al.  The Power of Pluralism for Automatic Program Synthesis , 1982, JACM.

[85]  Carl H. Smith,et al.  On the Inference of Programs Approximately Computing the Desired Function , 1986, AII.

[86]  Thomas Zeugmann,et al.  Inductive Inference of Almost Everywhere Correct Programs by Reliably Working Strategies , 1985, J. Inf. Process. Cybern..

[87]  Dominique Perrin,et al.  Finite Automata , 1958, Philosophy.

[88]  Giovanni Soda,et al.  Inductive Inference of Tree Automata by Recursive Neural Networks , 1997, AI*IA.

[89]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[90]  Carl H. Smith,et al.  Three Decades of Team Learning , 1994, AII/ALT.

[91]  Jude W. Shavlik,et al.  Machine Learning: Proceedings of the Fifteenth International Conference , 1998 .

[92]  Manfred K. Warmuth,et al.  The weighted majority algorithm , 1989, 30th Annual Symposium on Foundations of Computer Science.

[93]  Identifying Nearly Minimal Gödel Numbers From Additional Information , 1994, AII/ALT.

[94]  Thomas Zeugmann,et al.  A-posteriori Characterizations in Inductive Inference of Recursive Functions , 1983, J. Inf. Process. Cybern..

[95]  Thomas Zeugmann,et al.  One-Sided Error Probabilistic Inductive Inference and Reliable Frequency Identification , 1991, Inf. Comput..

[96]  Sanjay Jain.  Robust Behaviorally Correct Learning , 1999, Inf. Comput..

[97]  Frank Stephan,et al.  Learning classes of approximations to non-recursive function , 2002, Theor. Comput. Sci..

[98]  Charles X. Ling,et al.  Inductive Learning from Good Examples , 1991, IJCAI.

[99]  John Case,et al.  Incremental Concept Learning for Bounded Data Mining , 1997, Inf. Comput..

[100]  Klaus P. Jantke,et al.  Combining Postulates of Naturalness in Inductive Inference , 1981, J. Inf. Process. Cybern..

[101]  Mark A. Fulk,et al.  Saving the Phenomena: Requirements that Inductive Inference Machines Not Contradict Known Data , 1988, Inf. Comput..

[102]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[103]  Ulrike Brandt.  The Position of Index Sets of Identifiable Sets in the Arithmetical Hierarchy , 1986, Inf. Control..

[104]  Rolf Wiehagen,et al.  Learning and Consistency , 1995, GOSLER Final Report.

[105]  C. Dilworth.  Probability and Confirmation , 1988 .

[106]  J. Davenport.  Editor , 1960 .

[107]  Thomas Zeugmann,et al.  Learning indexed families of recursive languages from positive data: A survey , 2008, Theor. Comput. Sci..

[108]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[109]  R. Smullyan.  Theory of formal systems , 1962 .

[110]  Thomas Zeugmann,et al.  Consistency Conditions for Inductive Inference of Recursive Functions , 2006, JSAI.