Three ideal observer models for rule learning in simple languages

Children learning the inflections of their native language show the ability to generalize beyond the perceptual particulars of the examples they are exposed to. The phenomenon of "rule learning"--quick learning of abstract regularities from exposure to a limited set of stimuli--has become an important model system for understanding generalization in infancy. Experiments with adults and children have revealed differences in performance across domains and types of rules. To understand the representational and inferential assumptions necessary to capture this broad set of results, we introduce three ideal observer models for rule learning. Each model builds on the next, allowing us to test the consequences of individual assumptions. Model 1 learns a single rule, Model 2 learns a single rule from noisy input, and Model 3 learns multiple rules from noisy input. These models capture a wide range of experimental results--including several that have been used to argue for domain-specificity or limits on the kinds of generalizations learners can make-suggesting that these ideal observers may be a useful baseline for future work on rule learning.

[1]  R Sun,et al.  Proceedings of the 28th Annual Conference of the Cognitive Science Society , 2006 .

[2]  R N Aslin,et al.  Statistical Learning by 8-Month-Old Infants , 1996, Science.

[3]  S Pinker,et al.  Rules of language. , 1991, Science.

[4]  Lars Borin,et al.  Unsupervised Learning of Morphology , 2011, CL.

[5]  R. Gómez,et al.  Artificial grammar learning by 1-year-olds leads to specific and abstract knowledge , 1999, Cognition.

[6]  J. Mehler,et al.  Linguistic Constraints on Statistical Computations , 2005, Psychological science.

[7]  Donald J. Tyrrell,et al.  Learning and transfer of identity-difference relationships by infants* , 1993 .

[8]  John Hale,et al.  A Probabilistic Earley Parser as a Psycholinguistic Model , 2001, NAACL.

[9]  R N Henson,et al.  Mechanisms of top-down facilitation in perception of visual objects studied by FMRI. , 2007, Cerebral cortex.

[10]  D. Lewkowicz,et al.  Sequence learning in 4-month-old infants: do infants represent ordinal information? , 2009, Child development.

[11]  G. Altmann Learning and development in neural networks – the importance of prior experience , 2002, Cognition.

[12]  W. Geisler Ideal Observer Analysis , 2002 .

[13]  LouAnn Gerken,et al.  From domain-generality to domain-sensitivity: 4-Month-olds learn an abstract repetition rule in music that 7-month-olds do not , 2009, Cognition.

[14]  E. Newport,et al.  WORD SEGMENTATION : THE ROLE OF DISTRIBUTIONAL CUES , 1996 .

[15]  D. Luce,et al.  Object Detection and Recognition , 2009, Encyclopedia of Database Systems.

[16]  I. Biederman Perceiving Real-World Scenes , 1972, Science.

[17]  Thomas R. Shultz,et al.  Rule learning by Habituation can be Simulated in Neural Networks , 2020, Proceedings of the Twenty First Annual Conference of the Cognitive Science Society.

[18]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[19]  Peter M. Vishton,et al.  Rule learning by seven-month-old infants. , 1999, Science.

[20]  R. Gómez Variability and Detection of Invariant Structure , 2002, Psychological science.

[21]  LouAnn Gerken,et al.  Infants use rational decision criteria for choosing among models of their input , 2010, Cognition.

[22]  Eugene Galanter,et al.  Handbook of mathematical psychology: I. , 1963 .

[23]  L. Shastri Infants Learning Algebraic Rules , 1999, Science.

[24]  J. Tenenbaum,et al.  Poverty of the Stimulus? A Rational Approach , 2006 .

[25]  Ghislaine Dehaene-Lambertz,et al.  Perceptual constraints and the learnability of simple grammars , 2007, Cognition.

[26]  Joshua B. Tenenbaum,et al.  Learning and using relational theories , 2007, NIPS.

[27]  Thomas L. Griffiths,et al.  A Rational Analysis of Rule-Based Concept Learning , 2008, Cogn. Sci..

[28]  Diane Ohala,et al.  Contributions of phonetic token variability and word-type frequency to phonological representations. , 2011, Journal of child language.

[29]  Scott P. Johnson,et al.  Infant Rule Learning Facilitated by Speech , 2007, Psychological science.

[30]  M. Tomasello,et al.  Modeling children's early grammatical knowledge , 2009, Proceedings of the National Academy of Sciences.

[31]  R. Weale Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[32]  T. Griffiths,et al.  A Bayesian framework for word segmentation: Exploring the effects of context , 2009, Cognition.

[33]  Norbert Hornstein,et al.  Explanation in Linguistics: The Logical Problem of Language Acquisition , 1982 .

[34]  Kenneth D. Forbus,et al.  Modeling Infant Learning via Symbolic Structural Alignment , 2000 .

[35]  J. Mehler,et al.  Finding Words and Rules in a Speech Stream , 2008, Psychological science.

[36]  Michael C. Frank,et al.  Principles of Generalization for Learning Sequential Structure in Language , 2008 .

[37]  R. Levy Expectation-based syntactic comprehension , 2008, Cognition.

[38]  Christopher D. Manning,et al.  Probabilistic models of language processing and acquisition , 2006, Trends in Cognitive Sciences.

[39]  Linda B. Smith,et al.  Infants rapidly learn word-referent mappings via cross-situational statistics , 2008, Cognition.

[40]  Michael C. Frank,et al.  Modeling human performance in statistical word segmentation , 2010, Cognition.

[41]  R. Lickliter,et al.  Intersensory redundancy guides attentional selectivity and perceptual learning in infancy. , 2000, Developmental Psychology.

[42]  Javid Sadr,et al.  Object recognition and Random Image Structure Evolution , 2004, Cogn. Sci..

[43]  Peter Ford Dominey,et al.  Neural network processing of natural language: I. Sensitivity to serial, temporal and abstract structure of language in the infant , 2000 .

[44]  Dan Klein,et al.  Natural language grammar induction with a generative constituent-context model , 2005, Pattern Recognit..

[45]  Lynne Kiorpes,et al.  Ideal observer analysis of the development of spatial contrast sensitivity in macaque monkeys. , 2003, Journal of vision.

[46]  R. Gómez,et al.  Infant artificial language learning and language acquisition , 2000, Trends in Cognitive Sciences.

[47]  LouAnn Gerken,et al.  Three Exemplars Allow at Least Some Linguistic Generalizations: Implications for Generalization Mechanisms and Constraints , 2008 .

[48]  J. Saffran,et al.  Dog is a dog is a dog: Infant rule learning is not specific to language , 2007, Cognition.

[49]  Michael R. Brent,et al.  An Efficient, Probabilistically Sound Algorithm for Segmentation and Word Discovery , 1999, Machine Learning.

[50]  Donald J. Tyrrell,et al.  Perception of abstract identity/difference relationships by infants , 1991 .

[51]  LouAnn Gerken,et al.  Decisions, decisions: infant language learning when multiple generalizations are possible , 2006, Cognition.

[52]  Michael C. Frank,et al.  PSYCHOLOGICAL SCIENCE Research Article Using Speakers ’ Referential Intentions to Model Early Cross-Situational Word Learning , 2022 .

[53]  Carl E. Rasmussen,et al.  The Infinite Gaussian Mixture Model , 1999, NIPS.

[54]  B. Schölkopf,et al.  Edinburgh Research Explorer Interpolating between types and tokens by estimating power-law generators , 2006 .

[55]  Michael C. Frank,et al.  Overcoming Memory Limitations in Rule Learning , 2011 .

[56]  M. Tomasello Constructing a Language: A Usage-Based Theory of Language Acquisition , 2003 .

[57]  J. Mehler,et al.  Flexible Learning of Multiple Speech Structures in Bilingual Infants , 2009, Science.

[58]  K. McRae,et al.  Proceedings of the 30th Annual Conference of the Cognitive Science Society. , 2008 .

[59]  B. Hayes,et al.  Rules vs. analogy in English past tenses: a computational/experimental study , 2003, Cognition.

[60]  R. Gregory The intelligent eye , 1970 .

[61]  M. Srinivasan,et al.  The concepts of ‘sameness’ and ‘difference’ in an insect , 2001, Nature.

[62]  Michael C. Frank,et al.  Information from multiple modalities helps 5-month-olds learn abstract rules. , 2009, Developmental science.

[63]  L. Chalupa,et al.  The visual neurosciences , 2004 .

[64]  Esther Mondragón,et al.  Rule Learning by Rats , 2008, Science.

[65]  Robert Lickliter,et al.  Intersensory redundancy facilitates discrimination of tempo in 3-month-old infants. , 2002, Developmental psychobiology.

[66]  K. C. Anderson,et al.  Single neurons in prefrontal cortex encode abstract rules , 2001, Nature.

[67]  Morten H. Christiansen,et al.  The power of statistical learning: No need for algebraic rules , 2020, Proceedings of the Twenty First Annual Conference of the Cognitive Science Society.

[68]  M. Raijmakers Rethinking innateness: A connectionist perspective on development. , 1997 .

[69]  J. Tenenbaum,et al.  Generalization, similarity, and Bayesian inference. , 2001, The Behavioral and brain sciences.

[70]  John A. Goldsmith,et al.  Unsupervised Learning of the Morphology of a Natural Language , 2001, CL.

[71]  Michael C. Frank,et al.  Abstract Rule Learning for Visual Sequences in 8- and 11-Month-Olds. , 2009, Infancy : the official journal of the International Society on Infant Studies.

[72]  Thomas L. Griffiths,et al.  Adaptor Grammars: A Framework for Specifying Compositional Nonparametric Bayesian Models , 2006, NIPS.

[73]  J. Mehler,et al.  The neonate brain detects speech structure , 2008, Proceedings of the National Academy of Sciences.

[74]  Gert Webelhuth,et al.  Government and binding theory and the minimalist program : principles and parameters in syntactic theory , 1997 .

[75]  Jacques Mehler,et al.  The role of salience in the extraction of algebraic rules. , 2005, Journal of experimental psychology. General.

[76]  Jacques Mehler,et al.  Cognitive gains in 7-month-old bilingual infants , 2009, Proceedings of the National Academy of Sciences.