Integrating Graph-Based Vision Perception to Spoken Conversation in Human-Robot Interaction

In this paper we present the integration of graph-based visual perception to spoken conversation in human-robot interaction. The proposed architecture has a dialogue manager as the central component for the multimodal interaction, which directs the robot's behavior in terms of the intentions and actions associated to the conversational situations. We tested this ideas on a mobile robot programmed to act as a visitor's guide to our department of computer science.

[1]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Sven Wachsmuth,et al.  Using Speech in Visual Object Recognition , 2000, DAGM-Symposium.

[3]  Luis A. Pineda,et al.  Specification and interpretation of multimodal dialogue models for human-robot interaction , 2008 .

[4]  David G. Lowe,et al.  Shape indexing using approximate nearest-neighbour search in high-dimensional spaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Jesus A. Gonzalez,et al.  Advances in Artificial Intelligence – IBERAMIA 2004 , 2004, Lecture Notes in Computer Science.

[6]  Luis Villaseñor Pineda,et al.  DIMEx100: A New Phonetic and Speech Corpus for Mexican Spanish , 2004, IBERAMIA.

[7]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[8]  Trevor Darrell,et al.  Towards adaptive object recognition for situated human-computer interaction , 2007, WMISI '07.

[9]  Y. Kuno,et al.  Object recognition through human-robot interaction by speech , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[10]  Stepán Obdrzálek,et al.  Object recognition methods based on transformation covariant features , 2004, 2004 12th European Signal Processing Conference.

[11]  James F. Allen,et al.  An architecture for a generic dialogue shell , 2000, Natural Language Engineering.

[12]  Arturo Espinosa-Romero,et al.  A robust Graph Transformation Matching for non-rigid registration , 2009, Image Vis. Comput..