Neural networks for gesture-based remote control of a mobile robot

We present a neural network architecture for gesture-based interaction between a mobile robot and its user, thereby spanning a bridge from the localisation of the user over the recognition of its gestural instruction to the generation of the appropriate robot behavior. Since this system is applied under real-world conditions, especially concerning the localisation of a human user, some proper techniques are needed which have an adequate robustness. Hence, the combination of several components of saliency towards a multi-cue approach, integrating structure- and color-based features, is proposed. At the moment, the gestures themselves are very simple and can be described by the spatial relation between face and hands of the person. The organisation of the appropriate robot behavior is realised by means of a mixture of neural agents, responsible for certain aspects of the navigation task. Due to the complexity of the whole system, above all we use "standard neural network models", which are modified or extended according to the task at hand. Preliminary results show the reliability of the overall approach as well as the sufficient functionality of the already realised sub-modules.

[1]  James W. Davis,et al.  The Representation and Recognition of Action Using Temporal Templates , 1997, CVPR 1997.

[2]  Helge J. Ritter,et al.  Neural and Statistical Methods for Adaptive Color Segmentation - a Comparison , 1995, DAGM-Symposium.

[3]  H.-M. Gross,et al.  Extension of the ALVINN-architecture for robust visual guidance of a miniature robot , 1997, Proceedings Second EUROMICRO Workshop on Advanced Mobile Robots.

[4]  Dean A. Pomerleau,et al.  Neural Network Perception for Mobile Robot Guidance , 1993 .

[5]  Alex Waibel,et al.  Gaze Tracking Based on Face‐Color , 1995 .

[6]  Horst-Michael Groß,et al.  Biological neural architecture for chromatic adaptation resulting in constant color sensations , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[7]  Horst-Michael Groß,et al.  Neural Architecture for Gesture-Based Human-Machine-Interaction , 1997, Gesture Workshop.

[8]  Horst-Michael Groß,et al.  User localisation for visually-based human-machine-interaction , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[9]  Steven George Goodridge.  Multimedia sensor fusion for intelligent camera control and human-computer interaction , 1997 .

[10]  Helge J. Ritter,et al.  Erkennung menschlicher Kopfhaltungen mittels künstlicher neuronaler Netze , 1996, DAGM-Symposium.

[11]  David Kortenkamp,et al.  Recognizing and Interpreting Gestures on a Mobile Robot , 1996, AAAI/IAAI, Vol. 2.

[12]  Thomas Zahn,et al.  A PACED ANALOG SILICON MODEL OF AUDITORY ATTENTION , 1998 .

[13]  H. Martin Hunke,et al.  Locating and Tracking of Human Faces with Neural Networks , 1994 .

[14]  Aaron F. Bobick,et al.  Computers Seeing Action , 1996, BMVC.

[15]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[16]  Takeo Kanade,et al.  Human Face Detection in Visual Scenes , 1995, NIPS.

[17]  Hans-Joachim Böhme,et al.  Neuronale Verfahren zur Lokalisation und Bewertung von Handgesten in Real-World-Szenen , 1998 .