Hand Gesture Spotting Based on 3D Dynamic Features Using Hidden Markov Models

In this paper, we propose an automatic system that handles hand gesture spotting and recognition simultaneously in stereo color image sequences without any time delay based on Hidden Markov Models (HMMs). Color and 3D depth map are used to segment hand regions. The hand trajectory will determine in further step using Mean-shift algorithm and Kalman filter to generate 3D dynamic features. Furthermore, k-means clustering algorithm is employed for the HMMs codewords. To spot meaningful gestures accurately, a non-gesture model is proposed, which provides confidence limit for the calculated likelihood by other gesture models. The confidence measures are used as an adaptive threshold for spotting meaningful gestures. Experimental results show that the proposed system can successfully recognize isolated gestures with 98.33% and meaningful gestures with 94.35% reliability for numbers (0-9).

[1]  Nianjun Liu,et al.  Model structure selection & training algorithms for an HMM gesture recognition system , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[2]  Reinhard Klette,et al.  Computer vision - three-dimensional data from images , 1998 .

[3]  R. Oka,et al.  Spotting Recognition of Human Gestures from Motion Images , 1994 .

[4]  Ayoub Al-Hamadi,et al.  A Novel System for Automatic Hand Gesture Spotting and Recognition in Stereo Color Image , 2009, J. WSCG.

[5]  D.M. Mount,et al.  An Efficient k-Means Clustering Algorithm: Analysis and Implementation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[7]  Stan Sclaroff,et al.  Sign Language Spotting with a Threshold Model Based on Conditional Random Fields , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Daijin Kim,et al.  Simultaneous Gesture Segmentation and Recognition based on Forward Spotting Accumulative HMMs , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[9]  Ayoub Al-Hamadi,et al.  A Novel Method for 3D Face Detection and Normalization , 2007, J. Multim..

[10]  A. Al-Hamadi,et al.  Gesture Recognition for Alphabets from Hand Motion Trajectory Using Hidden Markov Models , 2007, 2007 IEEE International Symposium on Signal Processing and Information Technology.

[11]  T. Cover,et al.  Entropy, Relative Entropy and Mutual Information , 2001 .

[12]  Seong-Whan Lee,et al.  Gesture Spotting and Recognition for Human–Robot Interaction , 2007, IEEE Transactions on Robotics.

[13]  Jin-Hyung Kim,et al.  An HMM-Based Threshold Model Approach for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Chris H. Q. Ding,et al.  K-means clustering via principal component analysis , 2004, ICML.

[15]  Ayoub Al-Hamadi,et al.  Real-Time Capable System for Hand Gesture Recognition Using Hidden Markov Models in Stereo Color Image Sequences , 2008, J. WSCG.

[16]  Dorin Comaniciu,et al.  Kernel-Based Object Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[18]  Ho-Sub Yoon,et al.  Hand gesture recognition using combined features of location, angle and velocity , 2001, Pattern Recognit..

[19]  Deyou Xu.  A Neural Network Approach for Hand Gesture Recognition in Virtual Reality Driving Training System of SPG , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[20]  Ayoub Al-Hamadi,et al.  A Hidden Markov Model-based continuous gesture recognition system for hand motion trajectory , 2008, 2008 19th International Conference on Pattern Recognition.