Multimodal Communication for Human-Friendly Robot Partners in Informationally Structured Space

This paper proposes a multimodal communication method for human-friendly robot partners based on various types of sensors. First, we explain informationally structured space to extend the cognitive capabilities of robot partners based on environmental systems. Next, we discuss the suitable measurement range for recognition technologies of touch interface, voice recognition, human detection, gesture recognition, and others. Based on the suitable measurement ranges, we propose an integration method to estimate human behaviors based on the human detection using color image and 3-D distance information, and gesture recognition by the multilayered spiking neural network using the time series of human-hand positions. Furthermore, we propose a conversation system to realize the multimodal communication with a person. Finally, we show several experimental results of the proposed method, and discuss the future direction of this research.

[1]  Takenori Obo,et al.  Human localization based on spiking neural network in intelligent sensor networks , 2011, 2011 IEEE Workshop on Robotic Intelligence In Informationally Structured Space.

[2]  Jong-Hwan Kim.  The 3 rd Generation of Robotics: Ubiquitous Robot. , 2005 .

[3]  Kay Römer,et al.  Smart identification frameworks for ubiquitous computing applications , 2003, Proceedings of the First IEEE International Conference on Pervasive Computing and Communications, 2003. (PerCom 2003)..

[4]  Koen De Bosschere,et al.  Towards an Extensible Context Ontology for Ambient Intelligence , 2004, EUSAI.

[5]  Takenori Obo,et al.  Localization of human in informationally structured space based on sensor networks , 2010, International Conference on Fuzzy Systems.

[6]  John Anderson,et al.  Wireless sensor networks for habitat monitoring , 2002, WSNA '02.

[7]  Gaurav S. Sukhatme,et al.  Networked Infomechanical Systems (NIMS) for Ambient Intelligence , 2005, Ambient Intelligence.

[8]  Chrystopher L. Nehaniv,et al.  Using temporal information distance to locate sensorimotor experience in a metric space , 2005, 2005 IEEE Congress on Evolutionary Computation.

[9]  Burcin Becerik-Gerber,et al.  Performance-based evaluation of RFID-based indoor location sensing solutions for the built environment , 2011, Adv. Eng. Informatics.

[10]  Naoyuki Kubota,et al.  Information support system based on sensor networks , 2010, 2010 World Automation Congress.

[11]  Jing Shi,et al.  A comprehensive multi-factor analysis on RFID localization capability , 2011, Adv. Eng. Informatics.

[12]  Jerry Zhao,et al.  Habitat monitoring: application driver for wireless communications technology , 2001, CCRV.

[13]  Christopher J. Bishop,et al.  Pulsed Neural Networks , 1998 .

[14]  Fumio Kojima,et al.  Fuzzy and Neural Computing for Communication of a Partner Robot , 2003, J. Multiple Valued Log. Soft Comput..

[15]  Armando Fox,et al.  The Interactive Workspaces Project: Experiences with Ubiquitous Computing Rooms , 2002, IEEE Pervasive Comput..

[16]  Wulfram Gerstner,et al.  Spiking neurons , 1999 .

[17]  François Ingelrest,et al.  SensorScope: Application-specific sensor network for environmental monitoring , 2010, TOSN.

[18]  Naoyuki Kubota,et al.  Cooperative Perceptual Systems for Partner Robots Based on Sensor Network , 2006 .

[19]  Masahiro Tomono Shin.  Object-based Localization and Mapping using Loop Constraints and Geometric Prior Knowledge , 2003 .

[20]  N. Kubota,et al.  Structured learning for partner robots based on natural communication , 2008, 2008 IEEE Conference on Soft Computing in Industrial Applications.

[21]  Taysheng Jeng.  Toward a Ubiquitous Smart Space Design Framework , 2009, J. Inf. Sci. Eng..

[22]  Naoyuki Kubota.  Cognitive Development of Partner Robots Based on Interaction with People , 2008 .

[23]  Fumio Kojima,et al.  Multiple fuzzy state-value functions for human evaluation through interactive trajectory planning of a partner robot , 2006, Soft Comput..

[24]  Wulfram Gerstner,et al.  Spiking Neuron Models , 2002 .

[25]  Naoyuki Kubota,et al.  Topological environment reconstruction in informationally structured space for pocket robot partners , 2009, 2009 IEEE International Symposium on Computational Intelligence in Robotics and Automation - (CIRA).

[26]  Jong-Hwan Kim,et al.  The Third Generation of Robotics : Ubiquitous Robot , 2004 .

[27]  Naoyuki Kubota,et al.  Computational intelligence for structured learning of a partner robot based on imitation , 2005, Inf. Sci..