A Real Time Vision-Based Hand Gestures Recognition System

Hand gesture recognition is an important aspect in Human-Computer interaction, and can be used in various applications, such as virtual reality and computer games. In this paper, we propose a real time hand gesture recognition system. It includes three major procedures: detection, tracking and recognition. In hand detection stage, an open hand is detected by the histograms of oriented gradient and AdaBoost method. The hand detector is trained by the AdaBoost algorithm with HOG features. A contour based tracker is applied in combining condensation and partitioned sampling. After a hand is detected in the image, the tracker can track the hand contour in real time. During the tracking, the trajectory is saved to perform hand gesture recognition in the last stage. Recognition of the hand moving trajectory is implemented by hidden Markov models. Several HMMs are trained in advance, and the results from the tracking stage are then recognized using the trained HMMs. Experiments have been conducted to validate the performance of the proposed system. Under normal webcam it can recognize the predefined gestures quickly and precisely. As it is easy to develop other hand gestures, the proposed system has good potential in many applications.

[1]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[2]  Manolis I. A. Lourakis,et al.  Tracking of Human Hands and Faces through Probabilistic Fusion of Multiple Visual Cues , 2008, ICVS.

[3]  C. Taylor,et al.  Active shape models - 'Smart Snakes'. , 1992 .

[4]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[5]  Jin-Hyung Kim,et al.  An HMM-Based Threshold Model Approach for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  A. Corradini,et al.  Dynamic time warping for off-line recognition of a small gesture vocabulary , 2001, Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems.

[7]  Tieniu Tan,et al.  Real time hand tracking by combining particle filtering and mean shift , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[8]  Miaolong Yuan,et al.  Robust hand tracking using a simple color classification technique , 2008, VRCAI '08.

[9]  Allen R. Hanson,et al.  Computer Vision Systems , 1978 .

[10]  Lale Akarun,et al.  STARS: Sign tracking and recognition system using input-output HMMs , 2009, Pattern Recognit. Lett..

[11]  Nianjun Liu,et al.  Evaluation of HMM training algorithms for letter hand gesture recognition , 2003, Proceedings of the 3rd IEEE International Symposium on Signal Processing and Information Technology (IEEE Cat. No.03EX795).

[12]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  James M. Rehg,et al.  Statistical Color Models with Application to Skin Detection , 2004, International Journal of Computer Vision.

[14]  Geoffrey E. Hinton,et al.  Learning Generative Texture Models with extended Fields-of-Experts , 2009, BMVC.

[15]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[16]  Wu-Chih Hu,et al.  Vision-Based Hand Gesture Recognition Using PCA+Gabor Filters and SVM , 2009, 2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[17]  Mathias Kölsch,et al.  Robust hand detection , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[18]  Paul A. Viola,et al.  Fast Multi-view Face Detection , 2003 .