Towards Understanding Language through Perception in Situated Human-Robot Interaction: From Word Grounding to Grammar Induction

Robots are widely collaborating with human users in diferent tasks that require high-level cognitive functions to make them able to discover the surrounding environment. A difcult challenge that we briefy highlight in this short paper is inferring the latent grammatical structure of language, which includes grounding parts of speech (e.g., verbs, nouns, adjectives, and prepositions) through visual perception, and induction of Combinatory Categorial Grammar (CCG) for phrases. This paves the way towards grounding phrases so as to make a robot able to understand human instructions appropriately during interaction.

[1]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[2]  Daichi Mochihashi,et al.  A Probabilistic Approach to Unsupervised Induction of Combinatory Categorial Grammar in Situated Human-Robot Interaction , 2018, 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids).

[3]  Matthew R. Walter,et al.  Approaching the Symbol Grounding Problem with Probabilistic Graphical Models , 2011, AI Mag..

[4]  Tadahiro Taniguchi,et al.  Towards Understanding Object-Directed Actions: A Generative Model for Grounding Syntactic Categories of Speech Through Visual Perception , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[5]  Tadahiro Taniguchi,et al.  A generative framework for multimodal learning of spatial concepts and object categories: An unsupervised part-of-speech tagging and 3D visual perception based approach , 2017, 2017 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[6]  Tadahiro Taniguchi,et al.  Evaluation of Word Representations in Grounding Natural Language Instructions Through Computational Human-Robot Interaction , 2019, 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[7]  Yonatan Bisk,et al.  An HDP Model for Inducing Combinatory Categorial Grammars , 2013, TACL.

[8]  S. Griffis.  EDITOR , 1997, Journal of Navigation.

[9]  Yoshikatsu Hayashi,et al.  A probabilistic framework for comparing syntactic and semantic grounding of synonyms through cross-situational learning , 2018 .