Real-time dynamic hand gesture recognition using hidden Markov models

The goal of interaction between human and computer is to find a way to treat it like human-human interaction. Gestures play an important role in human's daily life in order to transfer data and human emotions. The gestures are results of part of body movement in which hand movement is the most widely used one that is known as dynamic hand gesture. So it is very important to follow and recognize hand motion to provide multi-purpose use. In this paper, we propose a system that recognizes hand gestures from continuous hand motion for English numbers from 0 to 9 in real-time, based on Hidden Markov Models (HMMs). There are two kinds of gestures, key gestures and link gestures. The link gestures are used to separate the key gestures from other hand motion trajectories (gesture path) that are called spotting. This type of spotting is a heuristic-based method that identifies start and end points of the key gestures. Then gesture path between these two points are given to HMMs for classification. Experimental results show that the proposed system can successfully recognize the key gestures with recognition rate of 93.84%and work in complex situations very well.

[1]  Seong-Whan Lee.  Automatic gesture recognition for intelligent human-robot interaction , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[2]  K. Balci,et al.  A Multimodal 3D Healthcare Communication System , 2007, 2007 3DTV Conference.

[3]  Heung-Il Suk,et al.  Hand gesture recognition based on dynamic Bayesian network framework , 2010, Pattern Recognit..

[4]  Kostas Karpouzis,et al.  SOMM: Self organizing Markov map for gesture recognition , 2010, Pattern Recognit. Lett..

[5]  Christoph Maggioni,et al.  A novel gestural input device for virtual reality , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[6]  Mubarak Shah,et al.  Visual gesture recognition , 1994 .

[7]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[8]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[9]  Chengdong Wu,et al.  Recognition of dynamic hand gesture based on SCHMM model , 2009, 2009 Chinese Control and Decision Conference.

[10]  John C. Hart,et al.  The CAVE: audio visual experience automatic virtual environment , 1992, CACM.

[11]  Ferdinand van der Heijden,et al.  Efficient adaptive density estimation per image pixel for the task of background subtraction , 2006, Pattern Recognit. Lett..

[12]  Kenji Suzuki,et al.  An efficient first-scan method for label-equivalence-based labeling algorithms , 2010, Pattern Recognit. Lett..

[13]  David Minnen,et al.  The perceptive workbench: Computer-vision-based gesture tracking, object tracking, and 3D reconstruction for augmented desks , 2003, Machine Vision and Applications.

[14]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[15]  Stan Sclaroff,et al.  A Unified Framework for Gesture Recognition and Spatiotemporal Gesture Segmentation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Agnès Just,et al.  A comparative study of two state-of-the-art sequence processing techniques for hand gesture recognition , 2009, Comput. Vis. Image Underst..

[17]  S. Marcel,et al.  A comparative study of two state-ofthe-art sequence processing techniques for hand gesture recognition , 2009 .

[18]  Mahmoud Othman Selim Mahmoud Elmezain,et al.  Hand gesture spotting and recognition using HMMs and CRFs in color image sequences , 2010 .

[19]  Hang Joon Kim,et al.  Vision-Based Game Interface Using Human Gesture , 2006, PSIVT.

[20]  Chin-Shyurng Fahn,et al.  A Human-Machine Interaction Technique: Hand Gesture Recognition Based on Hidden Markov Models with Trajectory of Hand Motion , 2011 .

[21]  James M. Rehg,et al.  Statistical Color Models with Application to Skin Detection , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).