Rule-based Hand Posture Recognition using Qualitative Finger Configurations Acquired with the Kinect

Gesture recognition systems exhibit failures when faced with large hand posture vocabularies or relatively new hand poses. One main reason is that 2D and 3D appearance-based approaches require significant amounts of training data. We address this problem by introducing a new 2D model-based approach to recognize hand postures. The model captures a high-level rule-based representation of the hand expressed in terms of finger poses and their qualitative configuration. The available 3D information is used to first segment the hand. We evaluate our approach on a Kinect dataset and report superior results while using less training data when comparing to state-of-the-art 3D SURF descriptor.

[1]  Luc Van Gool,et al.  Hough Transform and 3D SURF for Robust Three Dimensional Classification , 2010, ECCV.

[2]  Richard Bowden,et al.  Putting the pieces together: Connected Poselets for human pose estimation , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[3]  Andre L. C. Barczak,et al.  Real-time hand tracking using a set of cooperative classifiers based on Haar-like features , 2005 .

[4]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[5]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[6]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..

[7]  Nicolas Pugeault,et al.  Spelling it out: Real-time ASL fingerspelling recognition , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[8]  Anbumani Subramanian,et al.  Dynamic Hand Pose Recognition Using Depth Data , 2010, 2010 20th International Conference on Pattern Recognition.

[9]  Oguz Altun,et al.  Turkish fingerspelling recognition system using Generalized Hough Transform, interest regions, and local descriptors , 2011, Pattern Recognit. Lett..

[10]  Yosi Keller,et al.  Scale-Invariant Features for 3-D Mesh Models , 2012, IEEE Transactions on Image Processing.

[11]  Isabelle Guyon,et al.  ChaLearn gesture challenge: Design and first results , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[12]  Lale Akarun,et al.  Real time hand pose estimation using depth sensors , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[13]  Isabelle Guyon,et al.  Demonstrations and live evaluation for the gesture recognition challenge , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[14]  Luc Van Gool,et al.  Combining RGB and ToF cameras for real-time 3D hand gesture interaction , 2011, WACV.

[15]  Stephan Liwicki,et al.  Automatic recognition of fingerspelled words in British Sign Language , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[16]  Junsong Yuan,et al.  Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera , 2011, ACM Multimedia.

[17]  Ulrich Neumann,et al.  Real-time Hand Pose Recognition Using Low-Resolution Depth Images , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[18]  Junsong Yuan,et al.  Robust hand gesture recognition with kinect sensor , 2011, ACM Multimedia.