A Gesture Based Interface for Human-Robot Interaction

Service robotics is currently a highly active research area in robotics, with enormous societal potential. Since service robots directly interact with people, finding “natural” and easy-to-use user interfaces is of fundamental importance. While past work has predominately focussed on issues such as navigation and manipulation, relatively few robotic systems are equipped with flexible user interfaces that permit controlling the robot by “natural” means. This paper describes a gesture interface for the control of a mobile robot equipped with a manipulator. The interface uses a camera to track a person and recognize gestures involving arm motion. A fast, adaptive tracking algorithm enables the robot to track and follow a person reliably through office environments with changing lighting conditions. Two alternative methods for gesture recognition are compared: a template based approach and a neural network approach. Both are combined with the Viterbi algorithm for the recognition of gestures defined through arm motion (in addition to static arm poses). Results are reported in the context of an interactive clean-up task, where a person guides the robot to specific locations that need to be cleaned and instructs the robot to pick up trash.

[1]  Wolfram Burgard,et al.  Experiences with an Interactive Museum Tour-Guide Robot , 1999, Artif. Intell..

[2]  Horst-Michael Groß,et al.  User localisation for visually-based human-machine-interaction , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[3]  Jochen Triesch,et al.  Robotic Gesture Recognition , 1997, Gesture Workshop.

[4]  Michael J. Swain,et al.  An Architecture for Vision and Action , 1995, IJCAI.

[5]  Yuntao Cui,et al.  Hand sign recognition from intensity image sequences with complex backgrounds , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[6]  Michael J. Swain,et al.  Perseus: an extensible vision system for human-machine interaction , 1996 .

[7]  Michael J. Swain,et al.  Happy patrons make better tippers: creating a robot waiter using Perseus and the Animate Agent architecture , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[8]  David Kortenkamp,et al.  Recognizing and Interpreting Gestures on a Mobile Robot , 1996, AAAI/IAAI, Vol. 2.

[9]  R. James Firby.  Task Networks for Controlling Continuous Processes , 1994, AIPS.

[10]  Wolfram Burgard,et al.  MINERVA: a second-generation museum tour-guide robot , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[11]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Aaron F. Bobick,et al.  Learning visual behavior for gesture analysis , 1995, Proceedings of International Symposium on Computer Vision - ISCV.

[13]  Liqiang Feng,et al.  Navigating Mobile Robots: Systems and Techniques , 1996 .

[14]  Mark C. Torrance,et al.  Natural communication with robots , 1994 .

[15]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[16]  Alex Pentland,et al.  Invariant features for 3-D gesture recognition , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[17]  Robin R. Murphy,et al.  Artificial intelligence and mobile robots: case studies of successful robot systems , 1998 .

[18]  Sebastian Thrun,et al.  Learning Maps for Indoor Mobile Robot Navigation. , 1996 .

[19]  Wolfram Burgard,et al.  Map learning and high-speed navigation in RHINO , 1998 .

[20]  Satoru Hayamizu,et al.  Socially Embedded Learning of the Office-Conversant Mobil Robot Jijo-2 , 1997, IJCAI.

[21]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[22]  Sebastian Thrun,et al.  Learning to Locate an Object in 3D Space from a Sequence of Camera Images , 1998, ICML.

[23]  Alex Pentland,et al.  The ALIVE system: wireless, full-body interaction with autonomous agents , 1997, Multimedia Systems.

[24]  Alex Waibel,et al.  Readings in speech recognition , 1990 .

[25]  Ingemar J. Cox,et al.  Autonomous Robot Vehicles , 1990, Springer New York.

[26]  Aaron F. Bobick,et al.  Using Configuration States for the Representation and Recognition of Gesture , 2000 .

[27]  James L. Crowley.  Vision for man-machine interaction , 1995, Robotics Auton. Syst..

[28]  Michael J. Swain,et al.  Gesture recognition using the Perseus architecture , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  A. Brakensiek,et al.  Neural networks for gesture-based remote control of a mobile robot , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[30]  Justine Cassell,et al.  Recovering the temporal structure of natural gesture , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[31]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[32]  Dean A. Pomerleau,et al.  Neural Network Perception for Mobile Robot Guidance , 1993 .

[33]  Alberto Elfes,et al.  Occupancy grids: a probabilistic framework for robot perception and navigation , 1989 .

[34]  Sebastian Thrun,et al.  Template-Based Recognition of Pose and Motion Gestures On a Mobile Robot , 1998, AAAI/IAAI.

[35]  Alex Waibel,et al.  Tracking Human Faces in Real-Time, , 1995 .

[36]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Jochen Triesch,et al.  A gesture interface for human-robot-interaction , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[38]  J. Cohen,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulas , 1968 .

[39]  Gunther Wyszecki,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd Edition , 2000 .

[40]  Hans P. Moravec.  Sensor Fusion in Certainty Grids for Mobile Robots , 1988, AI Mag..

[41]  David Kortenkamp,et al.  Using stereo vision to pursue moving agents with a mobile robot , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[42]  Wendelin Feiten,et al.  Field test of a navigation system: autonomous cleaning in supermarkets , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[43]  Sebastian Thrun,et al.  Learning Metric-Topological Maps for Indoor Mobile Robot Navigation , 1998, Artif. Intell..

[44]  Trevor Darrell,et al.  Active face tracking and pose estimation in an interactive room , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[45]  Reid G. Simmons.  The 1994 AAAI Robot Competition and Exhibition , 1995, AI Mag..

[46]  David Kortenkamp,et al.  A mobile robot that recognizes people , 1995, Proceedings of 7th IEEE International Conference on Tools with Artificial Intelligence.

[47]  Carl F. R. Weiman,et al.  Helpmate autonomous mobile robot nav-igation system , 1991 .