Hand modeling, analysis and recognition

Analyzing hand gestures is a comprehensive task involving motion modeling, motion analysis, pattern recognition, machine learning and even psycholinguistic studies. A comprehensive review of various techniques in hand modeling, analysis, and recognition is needed. Due to the multidisciplinary nature of this research topic, we cannot include all the works in the literature. Rather than function as a thorough review paper, this article serves as a tutorial to this research topic. We study 3-D hand models, various articulated motion analysis methods, and gesture recognition techniques employed in current research. We conclude with some thoughts about future research directions. We also include some of our own research results, some of which are shown as examples.

[1]  Yangsheng Xu,et al.  Gesture interface: modeling and learning , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[2]  Mubarak Shah,et al.  Visual gesture recognition , 1994 .

[3]  Thomas S. Huang,et al.  Vision based hand modeling and tracking for virtual teleconferencing and telecollaboration , 1995, Proceedings of IEEE International Conference on Computer Vision.

[4]  D. McNeill.  Hand and Mind , 1995 .

[5]  J. Ohya,et al.  Applications of HMM modeling to recognizing human gestures in image sequences for a man-machine interface , 1995, Proceedings 4th IEEE International Workshop on Robot and Human Communication.

[6]  James L. Crowley,et al.  Finger Tracking as an Input Device for Augmented Reality , 1995 .

[7]  Tosiyasu L. Kunii,et al.  Model-based analysis of hand posture , 1995, IEEE Computer Graphics and Applications.

[8]  Takeo Kanade,et al.  Model-based tracking of self-occluding articulated objects , 1995, Proceedings of IEEE International Conference on Computer Vision.

[9]  Francis K. H. Quek.  Unencumbered Gestural Interaction , 1996, IEEE Multim..

[10]  Jochen Triesch,et al.  Robust classification of hand postures against complex backgrounds , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[11]  John R. Kender,et al.  Finding skin in color images , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[12]  Alex Pentland,et al.  Active gesture recognition using partially observable Markov decision processes , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[13]  David C. Hogg,et al.  Towards 3D hand tracking using a deformable model , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[14]  Yuntao Cui,et al.  Hand sign recognition from intensity image sequences with complex backgrounds , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[15]  Alex Pentland,et al.  Coupled hidden Markov models for complex action recognition , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Jake K. Aggarwal,et al.  Human motion analysis: a review , 1997, Proceedings IEEE Nonrigid and Articulated Motion Workshop.

[17]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Christoph Bregler,et al.  Learning and recognizing human dynamics in video sequences , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  S. Sarkar,et al.  Human skin and hand motion analysis from range image sequences using nonlinear FEM , 1997, Proceedings IEEE Nonrigid and Articulated Motion Workshop.

[21]  Michael J. Black,et al.  Analysis of gesture and action in technical talks for video indexing , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Vladimir Pavlovic,et al.  A visual computing environment for very large scale biomolecular modeling , 1997, Proceedings IEEE International Conference on Application-Specific Systems, Architectures and Processors.

[23]  James W. Davis,et al.  The representation and recognition of human movement using temporal templates , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Shaogang Gong,et al.  Colour Model Selection and Adaption in Dynamic Scenes , 1998, ECCV.

[25]  Aaron F. Bobick,et al.  Recognition and interpretation of parametric gesture , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[26]  Kang-Hyun Jo,et al.  Manipulative hand gesture recognition using task knowledge for human computer interaction , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[27]  Dimitris N. Metaxas,et al.  ASL recognition based on a coupling between HMMs and 3D motion analysis , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[28]  Alex Pentland,et al.  A Wearable Computer Based American Sign Language Recognizer , 1998, Assistive Technology and Artificial Intelligence.

[29]  Rajeev Sharma,et al.  Reliable tracking of human arm dynamics by multiple cue integration and constraint fusion , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[30]  Shan Lu,et al.  Color-based hands tracking system for sign language recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[31]  Yoshiaki Shirai,et al.  Hand gesture estimation and model refinement using monocular camera-ambiguity limitation by inequality constraints , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[32]  J. Cassell.  Computer Vision for Human–Machine Interaction: A Framework for Gesture Generation and Interpretation , 1998 .

[33]  Matthew Turk,et al.  View-based interpretation of real-time optical flow for gesture recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[34]  Andrew Blake,et al.  Classification of human body motion , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[35]  Ying Wu,et al.  Capturing articulated human hand motion: a divide-and-conquer approach , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[36]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[37]  Alex Pentland,et al.  Modeling and Prediction of Human Behavior , 1999, Neural Computation.

[38]  Vladimir Pavlovic,et al.  Dynamic bayesian networks for information fusion with applications to human-computer interfaces , 1999 .

[39]  Ying Wu,et al.  View-independent recognition of hand postures , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[40]  Ying Wu,et al.  An Adaptive Self-Organizing Color Segmentation Algorithm with Application to Robust Real-time Human Hand Localization , 2000 .

[41]  Francis K. H. Quek,et al.  Gesture, speech, and gaze cues for discourse segmentation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[42]  Rómer Rosales,et al.  Inferring body pose without tracking body parts , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[43]  Ying Wu,et al.  Color tracking by transductive learning , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).