Hand gesture recognition using RGB-D cues

In this paper, we propose a hand gesture recognition method in the clutter background by fusing the RGB-D cues. Since the hand localization is the key issue, we propose a coarse-to-fine procedure to detect hand accurately, which combines the statistic skin model using color information with depth prior knowledge. By detecting the skin candidate regions on the color image with Gaussian Mixture Model (GMM) skin model, hand region is obtained by compounding the depth information with the assumption that hands are at the closest position to the camera in all skin regions. Then, a new descriptor based on saliency point is used to represent the binary hand properly. A new hand model containing the wrist is proposed and the gesture recognition based on special points is applied. The experiment results demonstrate that our method performs better than NMI and moment based methods with a 96.2% recognition rate.

[1]  Miguel A. Ferrer,et al.  Biometric Identification Based on Hand-Shape Features Using a HMM Kernel , 2011, 2011 International Conference on Hand-Based Biometrics.

[2]  F. Wong,et al.  Hidden Markov Model-Based Gesture Recognition with Overlapping Hand-Head/Hand-Hand Estimated Using Kalman Filter , 2012, 2012 Third International Conference on Intelligent Systems Modelling and Simulation.

[3]  Lars Bretzner,et al.  Hand gesture recognition using multi-scale colour features, hierarchical models and particle filtering , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[4]  Ratika Pradhan,et al.  Hand Gesture Recognition : A Comparative Study , .

[5]  Yi Li,et al.  Research on 3D Hand Tracking Using Particle Filtering , 2008, 2008 Fourth International Conference on Natural Computation.

[6]  Nikolaos G. Bourbakis,et al.  A survey of skin-color modeling and detection methods , 2007, Pattern Recognit..

[7]  Ying Wu,et al.  Hand modeling, analysis and recognition , 2001, IEEE Signal Process. Mag..

[8]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[9]  Ho-Sub Yoon,et al.  Hand gesture recognition using combined features of location, angle and velocity , 2001, Pattern Recognit..

[10]  Seiji Inokuchi,et al.  Simulation and analysis of spectral distributions of human skin , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[11]  Pham The Bao,et al.  A New Approach to Hand Tracking and Gesture Recognition by a New Feature Type and HMM , 2009, 2009 Sixth International Conference on Fuzzy Systems and Knowledge Discovery.

[12]  Ho-Sub Yoon,et al.  Hand gesture recognition using hidden Markov models , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[13]  A. Kendon.  Gesture: Visible Action as Utterance , 2004 .

[14]  Thomas S. Huang,et al.  Vision based hand modeling and tracking for virtual teleconferencing and telecollaboration , 1995, Proceedings of IEEE International Conference on Computer Vision.

[15]  Maja Pantic,et al.  Human body gesture recognition using adapted auxiliary particle filtering , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[16]  Zhiquan Feng,et al.  A Particle Filtering for 3D Human Hand Tracking , 2008, 2008 The 9th International Conference for Young Computer Scientists.

[17]  Zhou Hang,et al.  An OPS Hand Tracking Algorithm Based on Optical Flow , 2009, 2009 WRI World Congress on Software Engineering.

[18]  S. J. Matcher,et al.  Computer simulation of the skin reflectance spectra , 2003, Comput. Methods Programs Biomed..

[19]  Chen Houjin,et al.  A new approach of hand tracking based on integrated optical flow analyse , 2010, IEEE 10th INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING PROCEEDINGS.

[20]  A. B. M. Shawkat Ali,et al.  HMM based hand gesture recognition: A review on techniques and approaches , 2009, 2009 2nd IEEE International Conference on Computer Science and Information Technology.

[21]  Xia Liu,et al.  Hand gesture recognition using depth data , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[22]  Shengtong Zhong,et al.  Hand Tracking by Particle Filtering with Elite Particles Mean Shift , 2008, 2008 Japan-China Joint Workshop on Frontier of Computer Science and Technology.