Hand number gesture recognition using recognized hand parts in depth images

In this paper, we present a novel approach of recognizing hand number gestures using the recognized hand parts in a depth image. Our proposed approach is divided into two stages: (i) hand parts recognition by random forests (RFs) and (ii) rule-based hand number gestures recognition. In the first stage, we create a database (DB) of synthetic hand depth silhouettes and their corresponding hand parts-labeled maps and then train RFs with the DB. Via the trained RFs, we recognize or label the hand parts in a depth silhouette. In the second stage, based on the information of the recognized or labeled hand parts, hand number gestures are recognized according to our derived rules. In our experiments, we quantitatively and qualitatively evaluated our hand parts recognition system with synthetic and real data. Then, we tested our hand number gesture recognition system with real data. Our results show the average recognition rate of 97.80 % over the ten hand number gestures from five different subjects.

[1]  S Padam Priyal,et al.  A study on static hand gesture recognition using moments , 2010, 2010 International Conference on Signal Processing and Communications (SPCOM).

[2]  Bhavsar Swapna,et al.  Hand Gesture Recognition System for Numbers Using Thresholding , 2011 .

[3]  Takayoshi Yamashita,et al.  Hand gesture based TV control system — Towards both user- & machine-friendly gesture applications , 2013, The 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision.

[4]  Vincent Lepetit,et al.  Randomized trees for real-time keypoint recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Alaa Barkoky,et al.  Static hand gesture recognition of Persian sign numbers using thinning method , 2011, 2011 International Conference on Multimedia Technology.

[6]  D. Menotti,et al.  Hu and Zernike Moments for Sign Language Recognition , 2012 .

[7]  Paolo Dario,et al.  A Survey of Glove-Based Systems and Their Applications , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[8]  Haitham Hasan,et al.  RETRACTED ARTICLE: Human–computer interaction using vision-based hand gesture recognition systems: a survey , 2013, Neural Computing and Applications.

[9]  Junsong Yuan,et al.  Robust Part-Based Hand Gesture Recognition Using Kinect Sensor , 2013, IEEE Transactions on Multimedia.

[10]  Juan M. Bogado,et al.  GLOVE-BASED GESTURE RECOGNITION SYSTEM , 2012 .

[11]  Chih-Hung Wu,et al.  Depth-based hand gesture recognition for home appliance control , 2013, 2013 IEEE International Symposium on Consumer Electronics (ISCE).

[12]  Francesco Camastra,et al.  Handy: A real-time three color glove-based gesture recognizer with learning vector quantization , 2012, Expert Syst. Appl..

[13]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[14]  T.D. Bui,et al.  Recognizing Postures in Vietnamese Sign Language With MEMS Accelerometers , 2007, IEEE Sensors Journal.

[15]  Xia Liu,et al.  Hand gesture recognition using depth data , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[16]  A. Al-Hamadi,et al.  Spatio-temporal feature extraction-based hand gesture recognition for isolated American Sign Language and Arabic numbers , 2009, 2009 Proceedings of 6th International Symposium on Image and Signal Processing and Analysis.

[17]  Liviu Goras,et al.  On HMM static hand gesture recognition , 2011, ISSCS 2011 - International Symposium on Signals, Circuits and Systems.

[18]  Jing Yang,et al.  Inertial sensor based recognition of 3-D character gestures with an ensemble classifiers , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[19]  R. S. Jadon,et al.  A REVIEW OF VISION BASED HAND GESTURES RECOGNITION , 2009 .

[20]  Németh Zoltán.  Autodesk 3ds Max , 2015 .

[21]  Prabin Kumar Bora,et al.  A robust static hand gesture recognition system using geometry based normalizations and Krawtchouk moments , 2013, Pattern Recognit..

[22]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[23]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[24]  Guido M. Cortelazzo,et al.  Hand gesture recognition with depth data , 2013, ARTEMIS '13.

[25]  Robin R. Murphy,et al.  Hand gesture recognition with depth images: A review , 2012, 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication.