Iterative Learning from Positive Data and Negative Counterexamples

A model for learning in the limit is defined where a (so-called iterative) learner gets all positive examples from the target language, tests every new conjecture with a teacher (oracle) if it is a subset of the target language (and if it is not, then it receives a negative counterexample), and uses only limited long-term memory (incorporated in conjectures). Three variants of this model are compared: when a learner receives least negative counterexamples, the ones whose size is bounded by the maximum size of input seen so far, and arbitrary ones. We also compare our learnability model with other relevant models of learnability in the limit, study how our model works for indexed classes of recursive languages, and show that learners in our model can work in non-U-shaped way - never abandoning the first right conjecture.

[1]  Carl H. Smith,et al.  The Power of Pluralism for Automatic Program Synthesis , 1982, JACM.

[2]  John Case,et al.  Machine Inductive Inference and Language Identification , 1982, ICALP.

[3]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[4]  M. Bowerman.  Starting to talk worse: Clues to language acquisition from children's late speech errors , 1982 .

[5]  Tatsuya Motoki,et al.  Inductive Inference from all Positive and Some Negative Data , 1991, Inf. Process. Lett..

[6]  John Case,et al.  Memory-Limited U-Shaped Learning , 2006, COLT.

[7]  John Case,et al.  U-shaped, iterative, and iterative-with-counter learning , 2007, Machine Learning.

[8]  Thomas Zeugmann,et al.  Incremental Learning from Positive Data , 1996, J. Comput. Syst. Sci..

[9]  Mark A. Fulk.  Prudence and Other Conditions on Formal Language Learning , 1990, Inf. Comput..

[10]  Klaus P. Jantke,et al.  Combining Postulates of Naturalness in Inductive Inference , 1981, J. Inf. Process. Cybern..

[11]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[12]  Paul Young,et al.  An introduction to the general theory of algorithms , 1978 .

[13]  Sandra Zilles,et al.  Comparison of Query Learning and Gold-Style Learning in Dependence of the Hypothesis Space , 2004, ALT.

[14]  Peter H. Schmitt,et al.  Nonmonotonic and Inductive Logic: 1st International Workshop Karlsruhe, Germany, December 4-7, 1990 Proceedings , 1991 .

[15]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[16]  Gary James Jason,et al.  The Logic of Scientific Discovery , 1988 .

[17]  Thomas Zeugmann,et al.  Types of monotonic language learning and their characterization , 1992, COLT '92.

[18]  Rolf Wiehagen.  A Thesis in Inductive Inference , 1990, Nonmonotonic and Inductive Logic.

[19]  Thomas Zeugmann,et al.  A Guided Tour Across the Boundaries of Learning Recursive Languages , 1995, GOSLER Final Report.

[20]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[21]  Rolf Wiehagen.  Limes-Erkennung rekursiver Funktionen durch spezielle Strategien , 1975, J. Inf. Process. Cybern..

[22]  Dana Angluin,et al.  Finding Patterns Common to a Set of Strings , 1980, J. Comput. Syst. Sci..

[23]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[24]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[25]  D. C. Cooper,et al.  Theory of Recursive Functions and Effective Computability , 1969, The Mathematical Gazette.

[26]  Chen Lin,et al.  Neural Information Processing -letters and Reviews Simplify Support Vector Machines by Iterative Learning , 2022 .

[27]  John Case,et al.  Language Learning with Some Negative Information , 1993, J. Comput. Syst. Sci..

[28]  Sanjay Jain,et al.  Learning Languages from Positive Data and Negative Counterexamples , 2004, ALT.

[29]  S. Pinker.  Formal models of language learning , 1979, Cognition.

[30]  Sanjay Jain,et al.  Learning Languages from Positive Data and a Finite Number of Queries , 2004, FSTTCS.

[31]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .