Neural recognition of human pointing gestures in real images

We present a neural network based system for the visual recognition of human hand pointing gestures from stereo pairs of video camera images. The accuracy of the current system allows to estimate the pointing target to an accuracy of 2 cm in a workspace area of 50×50 cm. The system consists of several neural networks that perform the tasks of image segmentation, estimation of hand location, estimation of 3D-pointing direction and necessary coordinate transforms. Drawing heavily on the use of learning algorithms, the functions of all network modules were created from data examples only.

[1]  Helge Ritter,et al.  Learning with the Self-Organizing Map , 1991 .

[2]  Helge J. Ritter,et al.  Neural and Statistical Methods for Adaptive Color Segmentation - a Comparison , 1995, DAGM-Symposium.

[3]  Helge J. Ritter,et al.  Rapid learning with parametrized self-organizing maps , 1996, Neurocomputing.

[4]  Helge Ritter,et al.  Learning 3D-shape perception with local linear maps , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[5]  Helge J. Ritter,et al.  Neural computation and self-organizing maps - an introduction , 1992, Computation and neural systems series.

[6]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[7]  Helge J. Ritter,et al.  A Hybrid Approach to Signal Interpretation Using Neuronal and Semantic Networks , 1993, DAGM-Symposium.

[8]  John A. Hertz,et al.  Exploiting Neurons with Localized Receptive Fields to Learn Chaos , 1990, Complex Syst..

[9]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[10]  Enno Littmann.  Strukturierung neuronaler Netze zwischen Biologie und Anwendung - biologische Modellierung, Kaskadierung und hybrider Ansatz , 1995, DISKI.

[11]  Franz Kummert,et al.  Recognition of 3D-Orientation from Monocular Color Images by Neural Semantic Networks , 1993 .

[12]  Franz Kummert,et al.  A hybrid system to detect hand orientation in stereo images , 1994 .

[13]  Mubarak Shah,et al.  Recognizing Hand Gestures , 1994, ECCV.

[14]  Farmer,et al.  Predicting chaotic time series. , 1987, Physical review letters.

[15]  T Poggio,et al.  Regularization Algorithms for Learning That Are Equivalent to Multilayer Networks , 1990, Science.

[16]  Alex Pentland,et al.  Classifying Hand Gestures with a View-Based Distributed Representation , 1993, NIPS.

[17]  Helge Ritter.  Parametrized Self-Organizing Maps for Vision Learning Tasks , 1994 .

[18]  John C. Platt,et al.  A Convolutional Neural Network Hand Tracker , 1994, NIPS.

[19]  Helge Ritter,et al.  Learning to recognize 3D-Hand Postures from Perspective Pixel Images , 1992 .