A model of language learning with semantics and meaning-preserving corrections

We present a computational model that takes into account semantics for language learning and allows us to model meaning-preserving corrections. The model is constructed with a learner and a teacher who interact in a sequence of shared situations by producing utterances intended to denote a unique object in each situation.We test our model with limited sublanguages of 10 natural languages exhibiting a variety of linguistic phenomena. The results show that learning to a high level of performance occurs after a reasonable number of interactions. Comparing the effect of a teacher who does no correction to that of a teacher who corrects whenever possible, we show that under certain conditions corrections can accelerate the rate of learning.We also define and analyze a simplified model of a probabilistic process of collecting corrections to help understand the possibilities and limitations of corrections in our setting.

[1]  Raymond J. Mooney,et al.  Learning Synchronous Grammars for Semantic Parsing with Lambda Calculus , 2007, ACL.

[2]  Terry Regier,et al.  The Human Semantic Potential: Spatial Language and Constrained Connectionism , 1996 .

[3]  Shalom Lappin,et al.  Linguistic Nativism and the Poverty of the Stimulus , 2011 .

[4]  Haim Kaplan,et al.  The greedy algorithm for edit distance with moves , 2006, Inf. Process. Lett..

[5]  Raymond J. Mooney,et al.  Unsupervised PCFG Induction for Grounded Language Learning with Highly Ambiguous Supervision , 2012, EMNLP.

[6]  Leonor Becerra-Bonache,et al.  Effects of Meaning-Preserving Corrections on Language Learning , 2011, CoNLL.

[7]  Brian Scassellati,et al.  Using sentence context and implicit contrast to learn sensor-grounded meanings for relational and deictic words: the twig system , 2008 .

[8]  Aline Villavicencio,et al.  Cognitive Aspects of Computational Language Acquisition , 2013, Theory and Applications of Natural Language Processing.

[9]  L L Travis,et al.  Limits on negative information in language input , 1989, Journal of Child Language.

[10]  Raymond J. Mooney,et al.  Learning to sportscast: a test of grounded language acquisition , 2008, ICML '08.

[11]  Jeffrey Mark Siskind,et al.  Lexical Acquisition in the Presence of Noise and Homonymy , 1994, AAAI.

[12]  A. M. Schaerlaekens,et al.  The two-word sentence in child language development : a study based on evidence provided by Dutch-speaking triplets , 1973 .

[13]  Leonor Becerra-Bonache,et al.  An Overview of How Semantics and Corrections Can Help Language Learning , 2011, 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology.

[14]  Andreas Stolcke,et al.  Miniature Language Acquisition: A Touchstone for Cognitive Science , 2002 .

[15]  R. Treiman,et al.  Brown & Hanlon revisited: mothers' sensitivity to ungrammatical forms , 1984, Journal of Child Language.

[16]  Makoto Kanazawa.  Learnable Classes of Categorial Grammars , 1998 .

[17]  Nick Chater,et al.  Distributional Information: A Powerful Cue for Acquiring Syntactic Categories , 1998, Cogn. Sci..

[18]  Jane C. Hill.  A computational model of language acquisition in the two-years-old , 1982 .

[19]  Shalom Lappin,et al.  Computational Learning Theory and Language Acquisition , 2012 .

[20]  Raymond J. Mooney,et al.  Training a Multilingual Sportscaster: Using Perceptual Context to Learn Language , 2014, J. Artif. Intell. Res..

[21]  Michael C. Frank,et al.  A Bayesian Framework for Cross-Situational Word-Learning , 2007, NIPS.

[22]  Brian Scassellati,et al.  A Robot That Uses Existing Vocabulary to Infer Non-Visual Word Meanings from Observation , 2007, AAAI.

[23]  Raymond J. Mooney,et al.  Generation by Inverting a Semantic Parser that Uses Statistical Machine Translation , 2007, NAACL.

[24]  Brian Scassellati,et al.  Robotic vocabulary building using extension inference and implicit contrast , 2009, Artificial Intelligence.

[25]  Susan Ervin-Tripp,et al.  SOME STRATEGIES FOR THE FIRST TWO YEARS , 1973 .

[26]  E. Clark,et al.  Adult reformulations of child errors as negative evidence , 2003, Journal of Child Language.

[27]  Chen Yu,et al.  The emergence of links between lexical acquisition and object categorization: a computational study , 2005, Connect. Sci..

[28]  Dana Angluin,et al.  Inference of Reversible Languages , 1982, JACM.

[29]  Richard M. De Morgan.  The programming language standards scene, ten years on Paper 16: C++ , 1994 .

[30]  C. Snow,et al.  Feedback to first language learners: the role of repetitions and clarification questions , 1986, Journal of Child Language.

[31]  Alfred V. Aho,et al.  The Transitive Reduction of a Directed Graph , 1972, SIAM J. Comput..

[32]  Terry Regier,et al.  The Emergence of Words: Attentional Learning in Form and Meaning , 2005, Cogn. Sci..

[33]  E. Veneziano.  Displacement and informativeness in child-directed talk , 2001 .

[34]  Carl de Marcken,et al.  Unsupervised language acquisition , 1996, ArXiv.

[35]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[36]  Alex Pentland,et al.  Learning words from sights and sounds: a computational model , 2002, Cogn. Sci..

[37]  Timo Knuutila,et al.  Polynomial Time Algorithms for Learning k -Reversible Languages and Pattern Languages with Correction Queries , 2007, ALT.

[38]  Leonor Becerra-Bonache,et al.  Learning Balls of Strings from Edit Corrections , 2008, J. Mach. Learn. Res..

[39]  Isabelle Tellier,et al.  Meaning Helps Learning Syntax , 1998, ICGI.

[40]  Eve V. Clark,et al.  The Lexicon in Acquisition , 1996 .

[41]  Yasubumi Sakakibara,et al.  Efficient Learning of Context-Free Grammars from Positive Structural Examples , 1992, Inf. Comput..

[42]  Dana H. Ballard,et al.  A multimodal learning interface for grounding spoken language in sensory perceptions , 2004, ACM Trans. Appl. Percept..

[43]  Leonor Becerra Bonache.  On the learnibility of Mildly Context-Sensitive languages using positive data and correction queries , 2006 .

[44]  J. Siskind.  A computational study of cross-situational techniques for learning word-to-meaning mappings , 1996, Cognition.

[45]  Leonor Becerra-Bonache,et al.  Learning Meaning Before Syntax , 2008, ICGI.

[46]  Afsaneh Fazly,et al.  A Probabilistic Computational Model of Cross-Situational Word Learning , 2010, Cogn. Sci..

[47]  Eve V. Clark,et al.  The principle of contrast: A constraint on language acquisition. , 1987 .

[48]  Efim B. Kinber,et al.  On Learning Regular Expressions and Patterns Via Membership and Correction Queries , 2008, ICGI.

[49]  Rohit J. Kate,et al.  Learning Language Semantics from Ambiguous Supervision , 2007, AAAI.

[50]  G. Marcus.  Negative evidence in language acquisition , 1993, Cognition.

[51]  Raymond J. Mooney,et al.  Generative Alignment and Semantic Parsing for Learning from Ambiguous Supervision , 2010, COLING.