Online learning and integration of complex action and word lexicons for language grounding

This paper presents a computational framework for the development and integration of action and language capabilities through symbol grounding. Our approach is based around the fundamental technique of building lexicons of perceptual “simulators” which link noisy sensory experiences to internal symbolic representations. These sensory representations are paired with a basic model of association, allowing for the grounding of linguistic symbols directly in action knowledge - a grounding which is then exploited to bootstrap the development of more advanced capabilities. The performance of this computational framework is tested in the context of online tutoring scenarios using the iCub robotic platform. Such an experimental environment requires development of techniques and algorithms suited for incremental learning and real-time processing.

[1]  L. Barsalou,et al.  Whither structured representation? , 1999, Behavioral and Brain Sciences.

[2]  Stan Davis,et al.  Comparison of Parametric Representations for Monosyllabic Word Recognition in Continuously Spoken Se , 1980 .

[3]  Yoshihiko Nakamura,et al.  Embodied Symbol Emergence Based on Mimesis Theory , 2004, Int. J. Robotics Res..

[4]  Giulio Sandini,et al.  The iCub humanoid robot: an open platform for research in embodied cognition , 2008, PerMIS.

[5]  Aude Billard,et al.  Incremental learning of gestures by imitation in a humanoid robot , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[6]  Angelo Cangelosi,et al.  An Embodied Model for Sensorimotor Grounding and Grounding Transfer: Experiments With Epigenetic Robots , 2006, Cogn. Sci..

[7]  Stephen E. Levinson,et al.  Hmm-Based Semantic Learning for a Mobile Robot , 2004 .

[8]  Giulio Sandini,et al.  In Press, Ieee Transactions on Autonomous Mental Development , 2010 .

[9]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[10]  Angelo Cangelosi,et al.  Integration of Speech and Action in Humanoid Robots: iCub Simulation Experiments , 2011, IEEE Transactions on Autonomous Mental Development.

[11]  Stevan Harnad.  The Symbol Grounding Problem , 1999, ArXiv.

[12]  Yoshihiko Nakamura,et al.  Humanoid Robot's Autonomous Acquisition of Proto-Symbols through Motion Segmentation , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[13]  Angelo Cangelosi,et al.  The grounding of higher order concepts in action and language: A cognitive robotics model , 2012, Neural Networks.

[14]  Jun Tani,et al.  Learning Semantic Combinatoriality from the Interaction between Linguistic and Behavioral Processes , 2005, Adapt. Behav..

[15]  Stephen E. Levinson,et al.  Mathematical Models for Speech Technology , 2005 .

[16]  Yoshihiko Nakamura,et al.  Statistically integrated semiotics that enables mutual inference between linguistic and behavioral symbols for humanoid robots , 2009, 2009 IEEE International Conference on Robotics and Automation.

[17]  Stephen E. Levinson,et al.  HMM-Based Concept Learning for a Mobile Robot , 2007, IEEE Transactions on Evolutionary Computation.

[18]  Gang George Yin,et al.  Recursive algorithms for estimation of hidden Markov models and autoregressive models with Markov regime , 2002, IEEE Trans. Inf. Theory.

[19]  Dana Kulic,et al.  Online Segmentation and Clustering From Continuous Observation of Whole Body Motions , 2009, IEEE Transactions on Robotics.

[20]  Michael P. Kaschak,et al.  Grounding language in action , 2002, Psychonomic bulletin & review.

[21]  B. Wrede,et al.  A self-referential childlike model to acquire phones, syllables and words from acoustic speech , 2008, 2008 7th IEEE International Conference on Development and Learning.