Real-Time Hand Gesture Recognition Based on the Depth Map for Human Robot Interaction

In this paper, we propose and implement a novel and real-time method for recognizing hand gestures using depth map. The depth map contains information relating to the distance of objects from a viewpoint. Microsoft’s Kinect sensor is used as input device to capture both the color image and its corresponding depth map. We first detect bare hand in cluttered background using distinct gray-level of the hand which is located near to the sensor. Then, scale invariance feature transform (SIFT) algorithm is used to extract feature vectors. Lastly, vocabulary tree along with K-means clustering method are used to partition the hand postures to ten simple sets as: "one", "two", "three", "four", "five", "six", "seven", "eight", "nine" and "ten" numbers based on the number of extended fingers. The vocabulary tree allows a larger and more discriminatory vocabulary to be used efficiently. Consequently, it leads to an improvement in accuracy of the clustering. The experimental results show superiority of the proposed method over other available approaches. With this approach, we are able to recognize 'numbers' gestures with over 90% accuracy. DOI: http://dx.doi.org/10.11591/ijece.v3i6.3954

[1]  Lars Bretzner,et al.  Hand gesture recognition using multi-scale colour features, hierarchical models and particle filtering , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[2]  Emil M. Petriu,et al.  A Prototype for 3-D Hand Tracking and Posture Estimation , 2008, IEEE Transactions on Instrumentation and Measurement.

[3]  N.D. Georganas,et al.  Real-time Vision-based Hand Gesture Recognition Using Haar-like Features , 2007, 2007 IEEE Instrumentation & Measurement Technology Conference IMTC 2007.

[4]  Yan Ke,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, CVPR 2004.

[5]  Thomas S. Huang,et al.  Tracking articulated hand motion with eigen dynamics analysis , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[6]  Sébastien Marcel,et al.  Hand posture recognition in a body-face centered space , 1999, CHI Extended Abstracts.

[7]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[8]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[9]  Paulo R. S. Mendonça,et al.  Model-based 3D tracking of an articulated hand , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[10]  S. Govindarajulu,et al.  A Comparison of SIFT, PCA-SIFT and SURF , 2012 .

[11]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[12]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[13]  Hanqing Lu,et al.  A real-time hand gesture recognition method , 2007, 2011 International Conference on Electronics, Communications and Control (ICECC).

[14]  Zhang Peng,et al.  An Automatic Hand Gesture Recognition System Based on Viola-Jones Method and SVMs , 2009, 2009 Second International Workshop on Computer Science and Engineering.

[15]  Yu Ren,et al.  Real-Time Hand Gesture Recognition Based on Vision , 2010, Edutainment.

[16]  Ying Wu,et al.  Capturing natural hand articulation , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[17]  Björn Stenger,et al.  Template-Based Hand Pose Recognition Using Multiple Cues , 2006, ACCV.

[18]  Chieh-Chih Wang,et al.  Hand posture recognition using adaboost with SIFT for human robot interaction , 2007 .

[19]  Manolis I. A. Lourakis,et al.  Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse , 2006, ECCV Workshop on HCI.

[20]  Nicolas D. Georganas,et al.  Real-Time Hand Gesture Detection and Recognition Using Bag-of-Features and Support Vector Machine Techniques , 2011, IEEE Transactions on Instrumentation and Measurement.