Results on memory-limited U-shaped learning

U-shaped learning is a learning behaviour in which the learner first learns a given target behaviour, then unlearns it and finally relearns it. Such a behaviour, observed by psychologists, for example, in the learning of past-tenses of English verbs, has been widely discussed among psychologists and cognitive scientists as a fundamental example of the non-monotonicity of learning. Previous theory literature has studied whether or not U-shaped learning, in the context of Gold's formal model of learning languages from positive data, is necessary for learning some tasks. It is clear that human learning involves memory limitations. In the present paper we consider, then, the question of the necessity of U-shaped learning for some learning models featuring memory limitations. Our results show that the question of the necessity of U-shaped learning in this memory-limited setting depends on delicate tradeoffs between the learner's ability to remember its own previous conjecture, to store some values in its long-term memory, to make queries about whether or not items occur in previously seen data and on the learner's choice of hypotheses space.

[1]  R. Stavy,et al.  U-shaped behavioral growth , 1982 .

[2]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[3]  Rolf Wiehagen A Thesis in Inductive Inference , 1990, Nonmonotonic and Inductive Logic.

[4]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[5]  Rolf Wiehagen,et al.  Learning and Consistency , 1995, GOSLER Final Report.

[6]  Daniel N. Osherson,et al.  Criteria of Language Learning , 1982, Inf. Control..

[7]  R. Soare Recursively enumerable sets and degrees , 1987 .

[8]  Klaus P. Jantke,et al.  Combining Postulates of Naturalness in Inductive Inference , 1981, J. Inf. Process. Cybern..

[9]  John R. Anderson,et al.  Why do children learn to say “Broke”? A model of learning the past tense without feedback , 2002, Cognition.

[10]  S Pinker,et al.  Overregularization in language acquisition. , 1992, Monographs of the Society for Research in Child Development.

[11]  Klaus P. Jantke Monotonic and non-monotonic inductive inference , 2009, New Generation Computing.

[12]  Jay Lash,et al.  Concepts of development , 1974 .

[13]  Jr. Hartley Rogers Theory of Recursive Functions and Effective Computability , 1969 .

[14]  Frank Stephan,et al.  Language Learning from Texts: Mind Changes, Limited Memory and Monotonicity (Extended Abstract). , 1995, COLT 1995.

[15]  Carl H. Smith,et al.  On the impact of forgetting on learning machines , 1995, JACM.

[16]  V. Marchman,et al.  U-shaped learning and frequency effects in a multi-layered perception: Implications for child language acquisition , 1991, Cognition.

[17]  John Case,et al.  Incremental Concept Learning for Bounded Data Mining , 1997, Inf. Comput..

[18]  M. Bowerman Starting to talk worse: Clues to language acquisition from children's late speech errors , 1982 .

[19]  John Case,et al.  The Power of Vacillation in Language Learning , 1999, SIAM J. Comput..

[20]  Susan Carey,et al.  7 – Face Perception: Anomalies of Development1 , 1982 .

[21]  John Case,et al.  U-shaped learning may be necessary 3 , 2004 .

[22]  John Case,et al.  Machine Inductive Inference and Language Identification , 1982, ICALP.

[23]  Carl H. Smith,et al.  Probabilistic versus Deterministic Memory Limited Learning , 1995, GOSLER Final Report.

[24]  John Case,et al.  U-Shaped, Iterative, and Iterative-with-Counter Learning , 2007, COLT.

[25]  P. Odifreddi Classical recursion theory , 1989 .

[26]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[27]  Sanjay Jain,et al.  Variations on U-Shaped Learning , 2005, COLT.

[28]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[29]  Kenneth Wexler,et al.  Formal Principles of Language Acquisition , 1980 .

[30]  John Case,et al.  When unlearning helps , 2008, Inf. Comput..

[31]  Rolf Wiehagen,et al.  Charakteristische Eigenschaften von erkennbaren Klassen rekursiver Funktionen , 1976, J. Inf. Process. Cybern..

[32]  Steffen Lange,et al.  Algorithmic Learning for Knowledge-Based Systems: Gosler Final Report , 1995 .

[33]  N. A. Lynch RELATIVIZATION OF THE THEORY OF COMPUTATION COMPLEXITY , 1972 .

[34]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[35]  Arun Sharma,et al.  On the Non-Existence of Maximal Inference Degrees for Language Identification , 1993, Inf. Process. Lett..

[36]  Sanjay Jain,et al.  Open Problems in "Systems That Learn" , 1994, J. Comput. Syst. Sci..

[37]  Thomas Zeugmann,et al.  Language learning in dependence on the space of hypotheses , 1993, COLT '93.

[38]  Rolf Wiehagen Limes-Erkennung rekursiver Funktionen durch spezielle Strategien , 1975, J. Inf. Process. Cybern..

[39]  Thomas Zeugmann,et al.  Incremental Learning from Positive Data , 1996, J. Comput. Syst. Sci..

[40]  John Case,et al.  Non U-Shaped Vacillatory and Team Learning , 2005, ALT.