Real-time Hand Gesture Recognition from Depth Images Using Convex Shape Decomposition Method

Hand gesture recognition is one of the most natural and intuitive ways to communicate between people and machines, since it closely mimics how human interact with each other. This paper presents a novel method for real-time markerless hand gesture recognition from depth images. The proposed method encompasses a collection of techniques that enable the detection, segmentation and recognition of hand gestures. A Hand detection and location method is employed using the depth information acquired from a depth sensor. Then, the hand is robustly segmented in cluttered background without any marker around. A convex shape decomposition method based on Radius Morse function is proposed for hand shape decomposition in real-time. Hand palm, fingertips and hand skeleton are recognized based on the hand shape decomposition and hand features. Moreover, we present a method for recognition of two-hand gestures. Representative experimental results demonstrate qualitatively and quantitatively that accurate hand gesture recognition can be achieved for real-time applications.

[1]  Hideo Saito,et al.  A Novel Inpainting-Based Layered Depth Video for 3DTV , 2011, IEEE Transactions on Broadcasting.

[2]  Yael Edan,et al.  Vision-based hand-gesture applications , 2011, Commun. ACM.

[3]  Tony Lindeberg,et al.  Feature Detection with Automatic Scale Selection , 1998, International Journal of Computer Vision.

[4]  Tobias Höllerer,et al.  Handy AR: Markerless Inspection of Augmented Reality Objects Using Fingertip Tracking , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[5]  Hanqing Lu,et al.  Hand Gesture Recognition Using Fast Multi-scale Analysis , 2007, Fourth International Conference on Image and Graphics (ICIG 2007).

[6]  Longin Jan Latecki,et al.  Path Similarity Skeleton Graph Matching , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Rod McCall,et al.  Lightweight palm and finger tracking for real-time 3D gesture control , 2011, 2011 IEEE Virtual Reality Conference.

[8]  Toby Sharp,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR.

[9]  Hanqing Lu,et al.  A real-time hand gesture recognition method , 2007, 2011 International Conference on Electronics, Communications and Control (ICECC).

[10]  Antonis A. Argyros,et al.  Efficient model-based 3D tracking of hand articulations using Kinect , 2011, BMVC.

[11]  David A. Forsyth,et al.  Generalizing motion edits with Gaussian processes , 2009, ACM Trans. Graph..

[12]  Long Quan,et al.  Image deblurring with blurred/noisy image pairs , 2007, SIGGRAPH 2007.

[13]  Manolis I. A. Lourakis,et al.  Real-Time Tracking of Multiple Skin-Colored Objects with a Possibly Moving Camera , 2004, ECCV.

[14]  Xiaofeng Mi,et al.  Separating Parts from 2D Shapes using Relatability , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[15]  Alexandru Telea,et al.  An Image Inpainting Technique Based on the Fast Marching Method , 2004, J. Graphics, GPU, & Game Tools.

[16]  Wenyu Liu,et al.  Convex shape decomposition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Haitao Yu,et al.  Real-Time Markerless Hand Gesture Recognition with Depth Camera , 2012, PCM.

[18]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, SIGGRAPH 2007.

[19]  Nassir Navab,et al.  Human skeleton tracking from depth data using geodesic distances and optical flow , 2012, Image Vis. Comput..

[20]  Longin Jan Latecki,et al.  Discrete Skeleton Evolution , 2007, EMMCVPR.

[21]  Nancy M. Amato,et al.  Approximate convex decomposition of polygons , 2006, Comput. Geom..

[22]  Tobias Höllerer,et al.  Multithreaded Hybrid Feature Tracking for Markerless Augmented Reality , 2009, IEEE Transactions on Visualization and Computer Graphics.

[23]  Lars Bretzner,et al.  Hand gesture recognition using multi-scale colour features, hierarchical models and particle filtering , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[24]  Hong Yang,et al.  An Improved Method of Wavelets Basis Image Denoising Using Besov Norm Regularization , 2007, Fourth International Conference on Image and Graphics (ICIG 2007).

[25]  Luc Van Gool,et al.  Combining RGB and ToF cameras for real-time 3D hand gesture interaction , 2011, WACV.

[26]  Junsong Yuan,et al.  Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera , 2011, ACM Multimedia.

[27]  Nassir Navab,et al.  Estimating human 3D pose from Time-of-Flight images based on geodesic distances and optical flow , 2011, Face and Gesture 2011.

[28]  Manolis I. A. Lourakis,et al.  Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse , 2006, ECCV Workshop on HCI.

[29]  Ruigang Yang,et al.  Fusion of time-of-flight depth and stereo for high accuracy depth maps , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Jesús Bescós,et al.  Real-time user independent hand gesture recognition from time-of-flight camera video using static and dynamic models , 2011, Machine Vision and Applications.

[31]  Zhengyou Zhang,et al.  3D Deformable Face Tracking with a Commodity Depth Camera , 2010, ECCV.

[32]  Jovan Popović,et al.  Real-time hand-tracking with a color glove , 2009, SIGGRAPH 2009.