Hand gesture recognition based on skeleton of point clouds

In this paper, we present a method of recognizing hand gestures in the form of point clouds recorded by Kinect sensor. Firstly, through Laplacian-based contraction and further processing, we extract skeleton points from point clouds of hands. Then, we apply a novel partition-based descriptor and corresponded algorithm to classify these skeletons and, taking one step further, to recognize gestures. In the process of recognition, the issue of scale variant and rotation variant are solved. Finally, to test and verify performance of our method, we design a series of experiments. Experimental results proved both its accuracy and robustness. Besides, we believe the skeleton-based way of recognition owns potential for further exploration.

[1]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[2]  Junjie Cao,et al.  Point Cloud Skeletons via Laplacian Based Contraction , 2010, 2010 Shape Modeling International Conference.

[3]  Helge J. Ritter,et al.  Visual recognition of continuous hand postures , 2002, IEEE Trans. Neural Networks.

[4]  Junsong Yuan,et al.  Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera , 2011, ACM Multimedia.

[5]  Stan Sclaroff,et al.  Translation and scale-invariant gesture recognition in complex scenes , 2008, PETRA '08.

[6]  Tamal K. Dey,et al.  Defining and computing curve-skeletons with medial geodesic function , 2006, SGP '06.

[7]  Junsong Yuan,et al.  Depth camera based hand gesture recognition and its applications in Human-Computer-Interaction , 2011, 2011 8th International Conference on Information, Communications & Signal Processing.

[8]  Junsong Yuan,et al.  Robust hand gesture recognition with kinect sensor , 2011, ACM Multimedia.

[9]  Narendra Ahuja,et al.  A potential-based generalized cylinder representation , 2004, Comput. Graph..

[10]  Xia Liu,et al.  Hand gesture recognition using depth data , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[11]  Serge J. Belongie,et al.  Matching with shape contexts , 2000, 2000 Proceedings Workshop on Content-based Access of Image and Video Libraries.

[12]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[13]  Seong-Whan Lee,et al.  View-independent human action recognition with Volume Motion Template on single stereo camera , 2010, Pattern Recognit. Lett..

[14]  Tong-Yee Lee,et al.  Skeleton extraction by mesh contraction , 2008, SIGGRAPH 2008.

[15]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.