Affordance based word-to-meaning association

This paper presents a method to associate meanings to words in manipulation tasks. We base our model on an affordance network, i.e., a mapping between robot actions, robot perceptions and the perceived effects of these actions upon objects. We extend the affordance model to incorporate words. Using verbal descriptions of a task, the model uses temporal co-occurrence to create links between speech utterances and the involved objects, actions and effects. We show that the robot is able form useful word-to-meaning associations, even without considering grammatical structure in the learning process and in the presence of recognition errors. These word-to-meaning associations are embedded in the robot's own understanding of its actions. Thus they can be directly used to instruct the robot to perform tasks and also allow to incorporate context in the speech recognition task.

[1]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[2]  Minoru Asada,et al.  Lexicon acquisition based on object-oriented behavior learning , 2006, Adv. Robotics.

[3]  Michael C. Frank,et al.  A Bayesian Framework for Cross-Situational Word-Learning , 2007, NIPS.

[4]  Eeva Klintfors,et al.  On the linguistic implications of context-bound adult-infant interactions , 2004 .

[5]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[6]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[7]  Manuel Lopes,et al.  Learning Object Affordances: From Sensory--Motor Coordination to Imitation , 2008, IEEE Transactions on Robotics.

[8]  Kris Demuynck,et al.  Discovering Phone Patterns in Spoken Utterances by , 2008 .

[9]  Chen Yu,et al.  A multimodal learning interface for grounding spoken language in sensory perceptions , 2003, ICMI '03.

[10]  Hugo Van hamme,et al.  Discovering Phone Patterns in Spoken Utterances by Non-Negative Matrix Factorization , 2008, IEEE Signal Processing Letters.

[11]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[12]  Chen Yu,et al.  A unified model of early word learning: Integrating statistical and social cues , 2007, Neurocomputing.

[13]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1998, Learning in Graphical Models.

[14]  L. Seabra Lopes,et al.  How many words can my robot learn?: An approach and experiments with one-class learning , 2007 .

[15]  John Juyang Weng.  The Developmental Approach to Intelligent Robots , 1998 .

[16]  Giampiero Salvi,et al.  Ecological language acquisition via incremental model-based clustering , 2005, INTERSPEECH.

[17]  N. Akhtar,et al.  Early lexical acquisition: the role of cross-situational learning , 1999 .

[18]  Hasegawa Osamu,et al.  Developmental Word Acquisition And Grammar Learning by Humanoid Robots through A Self-Organizing Incremental Neural Network , 2006 .

[19]  Giulio Sandini,et al.  Developmental robotics: a survey , 2003, Connect. Sci..

[20]  J. Siskind.  A computational study of cross-situational techniques for learning word-to-meaning mappings , 1996, Cognition.

[21]  Peter Ford Dominey,et al.  Learning Word Meaning and Grammatical Constructions from Narrated Video Events , 2003, HLT-NAACL 2003.