A Survey of Hand Posture and Gesture Recognition Techniques and Technology

This paper surveys the use of hand postures and gestures as a mechanism for interaction with computers, describing both the various techniques for performing accurate recognition and the technological aspects inherent to posture- and gesture-based interaction. First, the technological requirements and limitations for using hand postures and gestures are described by discussing both glove-based and vision-based recognition systems along with advantages and disadvantages of each. Second, the various types of techniques used in recognizing hand postures and gestures are compared and contrasted. Third, the applications that have used hand posture and gesture interfaces are examined. The survey concludes with a summary and a discussion of future research directions.

[1]  William T. Freeman,et al.  Television control by hand gestures , 1994 .

[2]  J. Michael Moshell,et al.  A Two-Handed Interface for Object Manipulation in Virtual Environments , 1995, Presence: Teleoperators & Virtual Environments.

[3]  Joseph J. LaViola.  A Multimodal Interface Framework for Using Hand Gestures and Speech in Virtual Environment Applications , 1999, Gesture Workshop.

[4]  Rich Gossweiler,et al.  Virtual Reality on Five Dollars a Day , 1991 .

[5]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[6]  G. Lakoff,et al.  Metaphors We Live by , 1981 .

[7]  Irfan Essa,et al.  Causal Analysis for Visual Gesture Understanding , 1995 .

[8]  Myron W. Krueger,et al.  Artificial reality II , 1991 .

[9]  Kunihiko Fukushima,et al.  Analysis of the process of visual pattern recognition by the neocognitron , 1989, Neural Networks.

[10]  Eugene Charniak,et al.  Statistical language learning , 1997 .

[11]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[12]  A. Lecours,et al.  The Biological foundations of gestures : motor and semiotic aspects , 1986 .

[13]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[14]  Jaron Lanier,et al.  A hand gesture interface device , 1987, CHI 1987.

[15]  Avinash C. Kak,et al.  Automatic learning of assembly tasks using a DataGlove system , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[16]  J. Cassell.  Computer Vision for Human–Machine Interaction: A Framework for Gesture Generation and Interpretation , 1998 .

[17]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[18]  Axel Pinz,et al.  Tracking in a multi-user augmented reality system , 1998 .

[19]  Justine Cassell,et al.  Recovering the temporal structure of natural gesture , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[20]  Mark A. Livingston,et al.  Superior augmented reality registration by integrating landmark tracking and magnetic tracking , 1996, SIGGRAPH.

[21]  J. P. Foley,et al.  Gesture and Environment , 1942 .

[22]  Alex Pentland,et al.  Recognition of Space-Time Gestures using a Distributed Representation , 1993 .

[23]  Steven D. Pieper,et al.  Hands-on interaction with virtual environments , 1989, UIST '89.

[24]  Alan Wexelblat,et al.  An approach to natural gesture in virtual environments , 1995, TCHI.

[25]  Steve Mann,et al.  Wearable Computing: A First Step Toward Personal Imaging , 1997, Computer.

[26]  W. Kadous.  GRASP: Recognition of Australian Sign Language Using Instrumented Gloves , 1995 .

[27]  Steve Bryson.  Virtual reality in scientific visualization , 1993, Comput. Graph..

[28]  R. Watson.  A Survey of Gesture Recognition Techniques , 1993 .

[29]  Jérôme Martin,et al.  An Appearance-Based Approach to Gesture-Recognition , 1997, ICIAP.

[30]  Ramjee Prasad,et al.  Hidden Markov models applied to on-line handwritten isolated character recognition , 1994, IEEE Trans. Image Process..

[31]  William Grimson,et al.  Object recognition by computer - the role of geometric constraints , 1991 .

[32]  Dean Rubine,et al.  Specifying gestures by example , 1991, SIGGRAPH.

[33]  C. Taylor,et al.  Active shape models - 'Smart Snakes'. , 1992 .

[34]  Craig A. Will,et al.  Review of Virtual Environment Interface Technology. , 1996 .

[35]  Andrew S. Glassner,et al.  Principles of Digital Image Synthesis , 1995 .

[36]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[37]  Joseph J. LaViola,et al.  FLEX AND PINCH: A CASE STUDY OF WHOLE HAND INPUT DESIGN FOR VIRTUAL ENVIRONMENT INTERACTION , 1999 .

[38]  Thomas B. Moeslund,et al.  Real-time recognition of hand alphabet gestures using principal component analysis , 1997 .

[39]  David Weimer,et al.  Interaction techniques using hand tracking and speech recognition , 1992 .

[40]  Biing-Hwang Juang,et al.  Hidden Markov Models for Speech Recognition , 1991 .

[41]  Frederick P. Brooks,et al.  Moving objects in space: exploiting proprioception in virtual-environment interaction , 1997, SIGGRAPH.

[42]  J. Drace,et al.  Evaluation of a fiber optic glove for semi-automated goniometric measurements. , 1990, Journal of rehabilitation research and development.

[43]  Shan Lu,et al.  The Recognition Algorithm with Non-contact for Japanese Sign Language Using Morphological Analysis , 1997, Gesture Workshop.

[44]  Ming Ouhyoung,et al.  A sign language recognition system using hidden markov model and context sensitive search , 1996, VRST.

[45]  KwangYun Wohn,et al.  Recognition of space-time hand-gestures using hidden Markov model , 1996, VRST.

[46]  David S. Broomhead,et al.  Multivariable Functional Interpolation and Adaptive Networks , 1988, Complex Syst..

[47]  D. Banarase,et al.  Hand posture recognition with the neocognitron network , 1993 .

[48]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[49]  Greg Welch,et al.  SCAAT: incremental tracking with incomplete information , 1997, SIGGRAPH.

[50]  Michel Beaudouin-Lafon,et al.  Charade: remote control of objects using free-hand gestures , 1993, CACM.

[51]  Geoffrey E. Hinton,et al.  Glove-TalkII: Mapping Hand Gestures to Speech Using Neural Networks , 1994, NIPS.

[52]  Philip Churchill,et al.  Sensing human hand motions for controlling dexterous robots , 1988 .

[53]  Michael Isard,et al.  3D position, attitude and shape input using video tracking of hands and lips , 1994, SIGGRAPH.

[54]  James W. Davis,et al.  Appearance-based motion recognition of human actions , 1996 .

[55]  Kishan G. Mehrotra,et al.  Elements of artificial neural networks , 1996 .

[56]  Brian Butterworth,et al.  Gesture and Silence as Indicators of Planning in Speech , 1978 .

[57]  Sharon L. Oviatt,et al.  Error resolution during multimodal human-computer interaction , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[58]  Kouichi Murakami,et al.  Gesture recognition using recurrent neural networks , 1991, CHI.

[59]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[60]  Beth Levy,et al.  Conceptual Representations in Lan-guage Activity and Gesture , 1980 .

[61]  Patrick van der Smagt,et al.  Introduction to neural networks , 1995, The Lancet.

[62]  Tomoichi Takahashi,et al.  Hand gesture coding based on experiments using a hand gesture interface device , 1991, SGCH.

[63]  Mubarak Shah,et al.  Establishing motion correspondence , 1991, CVGIP Image Underst..

[64]  Brian Everitt,et al.  Cluster analysis , 1974 .

[65]  Lawrence Birnbaum,et al.  Sensible Scenes: Visual Understanding of Complex Structures through Causal Analysis , 1993, AAAI.

[66]  David Zeltzer,et al.  A design method for “whole-hand” human-computer interaction , 1993, TOIS.

[67]  Markus Kohler,et al.  Special Topics of Gesture Recognition Applied in Intelligent Home Environments , 1997, Gesture Workshop.

[68]  R. Watson.  A Survey of Gesture RecognitionTechniques. , 1993 .

[69]  Shan Lu,et al.  Towards a Dialogue System Based on Recognition and Synthesis of Japanese Sign Language , 1997, Gesture Workshop.

[70]  Angie Legaspi,et al.  AMERICAN SOCIETY FOR SURGERY OF THE HAND , 1978, The Journal of bone and joint surgery. American volume.

[71]  Alan Wexelblat,et al.  A feature-based approach to continuous-gesture analysis , 1994 .

[72]  Fifth Dimension Technologies Training Solutions for Mining and Construction , .

[73]  Francis K. H. Quek,et al.  Toward a vision-based hand gesture interface , 1994 .

[74]  Mark E. Lucente,et al.  Visualization Space: A Testbed for Deviceless Multimodal User Interface , 1998 .

[75]  Geoffrey E. Hinton,et al.  Glove-TalkII: an adaptive gesture-to-formant interface , 1995, CHI '95.

[76]  Michael J. Papper,et al.  Using Gestures to Control a Virtual Arm , 1993, Virtual Reality Systems.

[77]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[78]  Richard Furuta,et al.  A logical hand device in virtual environments , 1994 .

[79]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[80]  Mubarak Shah,et al.  Establishing motion correspondence , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[81]  James W. Davis,et al.  GESTURE RECOGNITION , 2023, International Research Journal of Modernization in Engineering Technology and Science.

[82]  Neff Walker,et al.  Evaluation of the CyberGlove as a whole-hand input device , 1995, TCHI.

[83]  Takeo Kanade,et al.  DigitEyes: Vision-Based Human Hand Tracking , 1993 .

[84]  Brigitte Dorner,et al.  CHASING THE COLOUR GLOVE: VISUAL HAND TRACKING , 1994 .

[85]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[86]  Jack Sklansky,et al.  Estimating optical flow from clustered trajectories in velocity-time , 1992, [1992] Proceedings. 11th IAPR International Conference on Pattern Recognition.

[87]  Akira Utsumi.  Direct manipulation scene creation in 3D: estimating hand postures from multiple-camera images , 1997, SIGGRAPH '97.

[88]  Yangsheng Xu,et al.  Online, interactive learning of gestures for human/robot interfaces , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[89]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[90]  M. Brand.  Explanation-mediated vision: making sense of the world through causal analysis , 1995 .

[91]  Greg Welch,et al.  Welch & Bishop , An Introduction to the Kalman Filter 2 1 The Discrete Kalman Filter In 1960 , 1994 .