The Synthesis of Language Learners

Abstract An index for an r.e. class of languages (by definition) is a procedure which generates a sequence of grammars defining the class. An index for an indexed family of languages (by definition) is a procedure which generates a sequence of decision procedures defining the family. Studied is the metaproblem of synthesizing from indices for r.e. classes and for indexed families of languages various kinds of language learners for the corresponding classes or families indexed. Many positive results, as well as some negative results, are presented regarding the existence of such synthesizers. The negative results essentially provide lower bounds for the positive results. The proofs of some of the positive results yield, as pleasant corollaries, subset-principle or tell-tale style characterizations for the learnability of the corresponding classes or families indexed. For example, the indexed families of recursive languages that can be behaviorally correctly identified from positive data are surprisingly characterized by Angluin's condition 2 (the subset principle for circumventing overgeneralization).

[1]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[2]  Setsuo Ohsuga,et al.  Information Modelling and Knowledge Bases , 1990 .

[3]  Steffen Lange,et al.  Algorithmic Learning for Knowledge-Based Systems , 1995, Lecture Notes in Computer Science.

[4]  Thomas Zeugmann,et al.  Characterizations of Monotonic and Dual Monotonic Language Learning , 1995, Inf. Comput..

[5]  Paul Young,et al.  An introduction to the general theory of algorithms , 1978 .

[6]  Dick de Jongh,et al.  Angluin's theorem for indexed families of r.e. sets and applications , 1996, COLT '96.

[7]  James S. Royer.  A Connotational Theory of Program Structure , 1987, Lecture Notes in Computer Science.

[8]  Bala Kalyanasundaram,et al.  Breaking the probability ½ barrier in FIN-type learning , 1992, COLT '92.

[9]  Mark A. Fulk.  A study of inductive inference machines , 1986 .

[10]  Daniel N. Osherson,et al.  Synthesizing Inductive Expertise , 1988, Inf. Comput..

[11]  Leonard Pitt,et al.  Relations between probabilistic and team one-shot learners (extended abstract) , 1991, COLT '91.

[12]  M. B. Pour-El,et al.  A structural criterion for recursive enumeration without repetition , 1964 .

[13]  John Case,et al.  Synthesizing enumeration techniques for language learning , 1996, COLT '96.

[14]  Kenneth Wexler,et al.  Formal Principles of Language Acquisition , 1980 .

[15]  Robert Nix,et al.  Editing by example , 1985, POPL '84.

[16]  John Case,et al.  Synthesizing noise-tolerant language learners , 2001, Theor. Comput. Sci..

[17]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[18]  Daniel N. Osherson,et al.  Criteria of Language Learning , 1982, Inf. Control..

[19]  Mark A. Fulk.  Prudence and Other Conditions on Formal Language Learning , 1990, Inf. Comput..

[20]  Robert P. Daley.  Transformation of probabilistic learning strategies into deterministic learning strategies , 1988, COLT '88.

[21]  Thomas Zeugmann,et al.  One-Sided Error Probabilistic Inductive Inference and Reliable Frequency Identification , 1991, Inf. Comput..

[22]  Michael Stob,et al.  An analysis of a learning paradigm , 1986 .

[23]  Hiroki Arimura,et al.  Inductive Inference of Prolog Programs with Linear Data Dependency from Positive Data , 1993 .

[24]  David Kirsh,et al.  PDP Learnability and Innate Knowledge of Language , 1992 .

[25]  John Case,et al.  Learnability: Admissible, Co-Finite, and Hypersimple Languages , 1993, J. Comput. Syst. Sci..

[26]  Robert P. Daley.  Inductive inference hierarchies: probabilistic vs pluralistic strategies , 1985, Mathematical Methods of Specification and Synthesis of Software Systems.

[27]  Rusins Freivalds,et al.  On the Power of Probabilistic Strategies in Inductive Inference , 1984, Theor. Comput. Sci..

[28]  Robert E. Schapire,et al.  Design and analysis of efficient learning algorithms , 1992, ACM Doctoral dissertation award ; 1991.

[29]  Setsuo Arikawa,et al.  Pattern Inference , 1995, GOSLER Final Report.

[30]  Robert C. Berwick,et al.  The acquisition of syntactic knowledge , 1985 .

[31]  S. Kapur,et al.  Universal Grammar and Learnability Theory: The Case of Binding Domains and the ‘Subset Principle’ , 1993 .

[32]  Steven Davis,et al.  Connectionism: theory and practice , 1992 .

[33]  John Case,et al.  The Power of Vacillation in Language Learning , 1999, SIAM J. Comput..

[34]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[35]  Dana Angluin,et al.  Inference of Reversible Languages , 1982, JACM.

[36]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[37]  Carl H. Smith,et al.  Probability and Plurality for Aggregations of Learning Machines , 1987, Inf. Comput..

[38]  Thomas Zeugmann,et al.  Types of monotonic language learning and their characterization , 1992, COLT '92.

[39]  Leonard Pitt,et al.  Probabilistic inductive inference , 1989, JACM.

[40]  Daniel N. Osherson,et al.  Ideal Learning Machines , 1982, Cogn. Sci..

[41]  Ayumi Shinohara,et al.  Knowledge Acquisition from Amino Acid Sequences by Machine Learning System BONSAI , 1992 .

[42]  Mark A. Fulk.  Robust separations in inductive inference , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[43]  Hartley Rogers,et al.  Gödel numberings of partial recursive functions , 1958, Journal of Symbolic Logic.

[44]  D. C. Cooper,et al.  Theory of Recursive Functions and Effective Computability , 1969, The Mathematical Gazette.

[45]  Frank Stephan,et al.  Noisy Inference and Oracles , 1995, Theor. Comput. Sci..

[46]  Klaus P. Jantke.  Automatic synthesis of programs and inductive inference of functions , 1979, FCT.

[47]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[48]  John Case,et al.  Language Learning with Some Negative Information , 1993, J. Comput. Syst. Sci..

[49]  D. Osherson,et al.  A note on formal learning theory , 1982, Cognition.

[50]  S. Pinker.  Formal models of language learning , 1979, Cognition.

[51]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[52]  Thomas Zeugmann,et al.  A Guided Tour Across the Boundaries of Learning Recursive Languages , 1995, GOSLER Final Report.

[53]  K. Wexler.  On extensional learnability , 1982, Cognition.

[54]  John Case,et al.  Vacillatory and BC learning on noisy data , 1996, Theor. Comput. Sci..

[55]  S. Kapur.  Computational Learning of Languages , 1992 .

[56]  D. Osherson,et al.  Learning theory and natural language , 1984, Cognition.

[57]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[58]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[59]  Akihiro Yamamoto,et al.  Learning Elementary Formal Systems , 1992, Theor. Comput. Sci..

[60]  H. Putnam,et al.  Recursively enumerable classes and their application to recursive sequences of formal theories , 1965 .

[61]  Hilary Putnam,et al.  Trial and error predicates and the solution to a problem of Mostowski , 1965, Journal of Symbolic Logic.

[62]  Juris Viksna.  Probabilistic Inference of Approximations , 1991, Nonmonotonic and Inductive Logic.

[63]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[64]  John Case,et al.  Machine Inductive Inference and Language Identification , 1982, ICALP.

[65]  John Case,et al.  Learning Recursive Functions from Approximations , 1995, J. Comput. Syst. Sci..

[66]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[67]  K. Wexler.  The Subset Principle is an Intensional Principle , 1993 .

[68]  Rita M. Manzini,et al.  Parameters, binding theory and learnability , 1987 .

[69]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[70]  Bala Kalyanasundaram,et al.  Breaking the Probability 1/2 Barrier in FIN-Type Learning , 1995, J. Comput. Syst. Sci..

[71]  Leonard Pitt,et al.  A Characterization of Probabilistic Inference , 1984, FOCS.

[72]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[73]  T. Shinohara.  INFERRING UNIONS OF TWO PATTERN LANGUAGES , 1983 .

[74]  Thomas Zeugmann,et al.  Language learning in dependence on the space of hypotheses , 1993, COLT '93.

[75]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[76]  Keith Wright.  Identification of unions of languages drawn from an identifiable class , 1989, COLT '89.

[77]  N. Shapiro.  Review: E. Mark Gold, Limiting Recursion; Hilary Putnam, Trial and Error Predicates and the Solution to a Problem of Mostowski , 1971 .

[78]  Rusins Freivalds.  Recursiveness of the enumerating functions invreases the inferablity of recursively enumberable sets , 1985, Bull. EATCS.

[79]  Dana Angluin,et al.  Finding Patterns Common to a Set of Strings , 1980, J. Comput. Syst. Sci..

[80]  Gianfranco Bilardi,et al.  Language Learning Without Overgeneralization , 1992, Theor. Comput. Sci..

[81]  Yasuhito Mukouchi,et al.  Characterization of Finite Identification , 1992, AII.