Gesture Recognition using Neural Networks

Gesture interaction is a versatile, intuitive way of interacting with computers, especially suited for virtual environments and applications where many degrees of freedom are important. Human gestures have many meanings and uses, and are often di erent from person to person, while gesture input devices often introduce artifacts; this makes recognizing and interpreting gestures nontrivial. Gestures can be divided into postures, where the con guration of the hand is static, and true gestures, where the exion of the ngers and the hand position/direction are changing dynamically. A recognition model based on a hybrid arti cial neural network combining a radial basis function and a Bayesian classi er network is developed and tested with various forms of pre-processing or input representation. The results suggest that dynamic gesture recognition is feasible even for complex gestures, but that context information may be needed for reliable recognition.

[1]  Olof Hagsand,et al.  Interactive Multiuser VEs in the DIVE System , 1996, IEEE Multim..

[2]  Zbigniew Michalewicz,et al.  A Perspective on Evolutionary Computation , 1993, Evo Workshops.

[3]  Anders Lansner,et al.  A Higher order Bayesian Neural Network with Spiking Units , 1996, Int. J. Neural Syst..

[4]  Michel Beaudouin-Lafon,et al.  Charade: remote control of objects using free-hand gestures , 1993, CACM.

[5]  Kouichi Murakami,et al.  Gesture recognition using recurrent neural networks , 1991, CHI.

[6]  Örjan Ekeberg,et al.  A One-Layer Feedback Artificial Neural Network with a Bayesian Learning Rule , 1989, Int. J. Neural Syst..

[7]  Michael R. Berthold,et al.  A time delay radial basis function network for phoneme recognition , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[8]  Thoams Guthrie Zimmerman.  Personal area networks (PAN) : near-field intra-body communication , 1995 .

[9]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[10]  Naonori Ueda,et al.  A new competitive learning approach based on an equidistortion principle for designing optimal vector quantizers , 1994, Neural Networks.

[11]  Anders Lansner,et al.  A Bayesian Neural Network Model with Extensions a Bayesian Neural Network Model with Extensions , 1993 .

[12]  Dean Rubine,et al.  Specifying gestures by example , 1991, SIGGRAPH.

[13]  Ben Shneiderman,et al.  The future of interactive systems and the emergence of direct manipulation , 1982 .

[14]  Roy Kalawsky,et al.  The science of virtual reality and virtual environments - a technical, scientific and engineering reference on virtual environments , 1993 .

[15]  Myron W. Krueger,et al.  The emperor's new realities , 1993 .

[16]  Jiawei Hong,et al.  Calibrating a VPL DataGlove for teleoperating the Utah/MIT hand , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[17]  W. Kadous.  GRASP: Recognition of Australian Sign Language Using Instrumented Gloves , 1995 .

[18]  R. Watson.  A Survey of Gesture Recognition Techniques , 1993 .

[19]  Myron W. Krueger,et al.  Artificial reality II , 1991 .

[20]  Kai-Mikael Jää-Aro.  Three-dimensional Interaction and Presentation for Computer-Supported Cooperative Work , 1996 .

[21]  Alexander H. Waibel,et al.  Improving connected letter recognition by lipreading , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[22]  David Weimer,et al.  Interaction techniques using hand tracking and speech recognition , 1992 .

[23]  H. Abarbanel,et al.  Determining embedding dimension for phase-space reconstruction using a geometrical construction. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[24]  Yasuhito Suenaga,et al.  “FingeRing”: a full-time wearable interface , 1994, CHI '94.

[25]  KwangYun Wohn,et al.  Recognition of space-time hand-gestures using hidden Markov model , 1996, VRST.

[26]  Lucy Pao,et al.  Transformation of human hand positions for robotic hand control , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[27]  Charles Leave.  Neural Networks: Algorithms, Applications and Programming Techniques , 1992 .

[28]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[29]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[30]  C. Creider.  Hand and Mind: What Gestures Reveal about Thought , 1994 .

[31]  Q. Tuan Pham,et al.  Competitive Evolution: A Natural Approach to Operator Selection , 1993, Evo Workshops.

[32]  I. C. Wand,et al.  A.P.I.C. Studies in Data Processing , 1989 .

[33]  Steven D. Pieper,et al.  Hands-on interaction with virtual environments , 1989, UIST '89.

[34]  Russell Beale,et al.  Recognising postures and gestures using neural networks , 1992 .

[35]  Steve Mann,et al.  Wearable Computing: A First Step Toward Personal Imaging , 1997, Computer.

[36]  Klaus Boehm,et al.  Dynamic gesture recognition using neural networks: a fundament for advanced interaction construction , 1994, Electronic Imaging.

[37]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[38]  Sun-Yuan Kung,et al.  A delay damage model selection algorithm for NARX neural networks , 1997, IEEE Trans. Signal Process..

[39]  Joshua R. Smith.  Field Mice: Extracting Hand Geometry from Electric Field Measurements , 1996, IBM Syst. J..

[40]  Jaron Lanier,et al.  A hand gesture interface device , 1987, CHI 1987.

[41]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[42]  Alan Wexelblat.  Natural gesture in virtual environments , 1994 .

[43]  Jim Humphries,et al.  Gestural interaction in a virtual environment , 1994, Electronic Imaging.

[44]  Andrew D. Wilson,et al.  Using Connguration States for the Representation and Recognition of Gesture 2 Related Work , 1995 .

[45]  D. Banarase,et al.  Hand posture recognition with the neocognitron network , 1993 .