Automatic Learning from Positive Data and Negative Counterexamples

A model for learning in the limit is defined where a (so-called iterative) learner gets all positive examples from the target language, tests every new conjecture with a teacher (oracle) if it is a subset of the target language (and if it is not, then it receives a negative counterexample), and uses only limited long-term memory (incorporated in conjectures). Three variants of this model are compared: when a learner receives least negative counterexamples, the ones whose size is bounded by the maximum size of input seen so far, and arbitrary ones. We also compare our learnability model with other relevant models of learnability in the limit, study how our model works for indexed classes of recursive languages, and show that learners in our model can work in non-U-shaped way — never abandoning the first right conjecture.

[1]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[2]  Sanjay Jain,et al.  ON AUTOMATIC FAMILIES , 2011 .

[3]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[4]  Thomas Zeugmann,et al.  A Guided Tour Across the Boundaries of Learning Recursive Languages , 1995, GOSLER Final Report.

[5]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[6]  Mark A. Fulk.  Prudence and Other Conditions on Formal Language Learning , 1990, Inf. Comput..

[7]  Sanjay Jain,et al.  Learning languages from positive data and negative counterexamples , 2008, J. Comput. Syst. Sci..

[8]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[9]  John Case,et al.  Language Learning with Some Negative Information , 1993, J. Comput. Syst. Sci..

[10]  Achim Blumensath,et al.  Automatic structures , 2000, Proceedings Fifteenth Annual IEEE Symposium on Logic in Computer Science (Cat. No.99CB36332).

[11]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[12]  Thomas Zeugmann,et al.  Incremental Learning from Positive Data , 1996, J. Comput. Syst. Sci..

[13]  Karl R. Popper.  The Logic of Scientific Discovery. , 1977 .

[14]  John Case,et al.  Machine Inductive Inference and Language Identification , 1982, ICALP.

[15]  M. Bowerman.  Starting to talk worse: Clues to language acquisition from children's late speech errors , 1982 .

[16]  Dana Angluin,et al.  Finding Patterns Common to a Set of Strings , 1980, J. Comput. Syst. Sci..

[17]  Sanjay Jain,et al.  Learnability of Automatic Classes , 2010, LATA.

[18]  Thomas Zeugmann,et al.  Language learning in dependence on the space of hypotheses , 1993, COLT '93.

[19]  Robert H. Sloan,et al.  BOOK REVIEW: "SYSTEMS THAT LEARN: AN INTRODUCTION TO LEARNING THEORY, SECOND EDITION", SANJAY JAIN, DANIEL OSHERSON, JAMES S. ROYER and ARUN SHARMA , 2001 .

[20]  Tatsuya Motoki,et al.  Inductive Inference from all Positive and Some Negative Data , 1991, Inf. Process. Lett..

[21]  Anil Nerode,et al.  Automatic Presentations of Structures , 1994, LCC.

[22]  R. Treiman,et al.  Brown & Hanlon revisited: mothers' sensitivity to ungrammatical forms , 1984, Journal of Child Language.

[23]  Rolf Wiehagen.  A Thesis in Inductive Inference , 1990, Nonmonotonic and Inductive Logic.

[24]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[25]  Sandra Zilles,et al.  Comparison of Query Learning and Gold-Style Learning in Dependence of the Hypothesis Space , 2004, ALT.

[26]  S. Pinker.  Formal models of language learning , 1979, Cognition.

[27]  Sanjay Jain,et al.  Iterative Learning from Positive Data and Negative Counterexamples , 2007, ALT.

[28]  Sanjay Jain,et al.  Learning Languages from Positive Data and Negative Counterexamples , 2004, ALT.

[29]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[30]  Rolf Wiehagen.  Limes-Erkennung rekursiver Funktionen durch spezielle Strategien , 1975, J. Inf. Process. Cybern..