Gesture-based human-robot interaction using a knowledge-based software platform

Purpose – Achieving natural interactions by means of vision and speech between humans and robots is one of the major goals that many researchers are working on. This paper aims to describe a gesture‐based human‐robot interaction (HRI) system using a knowledge‐based software platform.Design/methodology/approach – A frame‐based knowledge model is defined for the gesture interpretation and HRI. In this knowledge model, necessary frames are defined for the known users, robots, poses, gestures and robot behaviors. First, the system identifies the user using the eigenface method. Then, face and hand poses are segmented from the camera frame buffer using the person's specific skin color information and classified by the subspace method.Findings – The system is capable of recognizing static gestures comprised of the face and hand poses, and dynamic gestures of face in motion. The system combines computer vision and knowledge‐based approaches in order to improve the adaptability to different people.Originality/val...

[1]  Haruki Ueno,et al.  Software Platform for Symbiotic Operations of Human and Networked Robots , 2001 .

[2]  Haruki Ueno,et al.  On Tracking of Eye for Human-Robot Interface , 2004, Int. J. Robotics Autom..

[3]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Ying Dai,et al.  Face-texture model based on SGLD and its application in face detection in a color scene , 1996, Pattern Recognit..

[5]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[7]  Haruki Ueno,et al.  A Knowledge-Based Information Modeling for Autonomous Humanoid Service Robot , 2002 .

[8]  Haruki Ueno,et al.  Face Detection and Facial Feature Localization for Human-machine Interface , 2003 .

[9]  Thomas S. Huang,et al.  Human face detection in a complex background , 1994, Pattern Recognit..

[10]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Akira Utsumi,et al.  Hand detection and tracking using pixel value distribution model for multiple-camera-based gesture interactions , 2002, Proceedings. IEEE Workshop on Knowledge Media Networking.

[12]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[13]  Haruki Ueno,et al.  SPAK: Software Platform for Agents and Knowledge Systems in Symbiotic Robots , 2004, IEICE Trans. Inf. Syst..

[14]  Sebastian Thrun,et al.  A Gesture Based Interface for Human-Robot Interaction , 2000, Auton. Robots.

[15]  Takahiro Watanabe,et al.  Real time gesture recognition using eigenspace from multi-input image sequences , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[16]  Tao Zhang,et al.  Human-robot interaction control for industrial robot arm through software platform for agents and knowledge management , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[17]  Tao Zhang,et al.  Face and Gesture Recognition Using Subspace Method for Human-Robot Interaction , 2004, PCM.

[18]  Saad Ahmed Sirohey,et al.  Human Face Segmentation and Identification , 1998 .

[19]  Avi Pfeffer,et al.  Probabilistic Frame-Based Systems , 1998, AAAI/IAAI.

[20]  Tao Zhang,et al.  Gesture Recognition for Human-Robot Interaction Through a Knowledge Based Software Platform , 2004, ICIAR.

[21]  R. E. Axtell.  The Do's and Taboos of Hosting International Visitors , 1990 .

[22]  Tao Zhang,et al.  Real-time Vision-based Gesture Recognition for Human Robot Interaction , 2004, 2004 IEEE International Conference on Robotics and Biomimetics.

[23]  Illah R. Nourbakhsh,et al.  A survey of socially interactive robots , 2003, Robotics Auton. Syst..

[24]  Marvin Minsky,et al.  A framework for representing knowledge , 1974 .

[25]  Gerhard Rigoll,et al.  High Performance Real-Time Gesture Recognition Using Hidden Markov Models , 1997, Gesture Workshop.