Hierarchically constrained 3D hand pose estimation using regression forests from single frame depth data

We apply Random Forests for regression (RDF-R) to 3D Hand Pose Estimation.RDF-R is generalized by hierarchical mode selection using constraints (RDF-R+).We test Classification Forests (RDF-C), RDF-R and RDF-R+ with 4 different datasets.The proposed method, RDF-R+ outperforms RDF-C and RDF-R. The emergence of inexpensive 2.5D depth cameras has enabled the extraction of the articulated human body pose. However, human hand skeleton extraction still stays as a challenging problem since the hand contains as many joints as the human body model. The small size of the hand also makes the problem more challenging due to resolution limits of the depth cameras. Moreover, hand poses suffer from self-occlusion which is considerably less likely in a body pose. This paper describes a scheme for extracting the hand skeleton using random regression forests in real-time that is robust to self- occlusion and low resolution of the depth camera. In addition to that, the proposed algorithm can estimate the joint positions even if all of the pixels related to a joint are out of the camera frame. The performance of the new method is compared to the random classification forests based method in the literature. Moreover, the performance of the joint estimation is further improved using a novel hierarchical mode selection algorithm that makes use of constraints imposed by the skeleton geometry. The performance of the proposed algorithm is tested on datasets containing synthetic and real data, where self-occlusion is frequently encountered. The new algorithm which runs in real time using a single depth image is shown to outperform previous methods.

[1]  Stavros J. Perantonis,et al.  Hand Shape and 3D Pose Estimation Using Depth Data from a Single Cluttered Frame , 2012, ISVC.

[2]  Xia Liu,et al.  Hand gesture recognition using depth data , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[3]  Andrew W. Fitzgibbon,et al.  Efficient regression of general-activity human poses from depth images , 2011, 2011 International Conference on Computer Vision.

[4]  Anbumani Subramanian,et al.  Dynamic Hand Pose Recognition Using Depth Data , 2010, 2010 20th International Conference on Pattern Recognition.

[5]  Hedvig Kjellström,et al.  Inferring hand pose: A comparative study of visual shape features , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[6]  Daniel Thalmann,et al.  Hand pose estimation by combining fingertip tracking and articulated ICP , 2012, VRCAI '12.

[7]  Nicolas Pugeault,et al.  Spelling it out: Real-time ASL fingerspelling recognition , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[8]  Yuan Yao,et al.  Real-Time Hand Pose Estimation from RGB-D Sensor , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[9]  Jeffrey Scott Vitter,et al.  Random sampling with a reservoir , 1985, TOMS.

[10]  Michael G. Strintzis,et al.  Real-time hand posture recognition using range data , 2008, Image Vis. Comput..

[11]  Chan-Su Lee,et al.  Articulated hand configuration and rotation estimation using extended torus manifold embedding , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[12]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[13]  Lale Akarun,et al.  Real time hand pose estimation using depth sensors , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[14]  Luigi Gallo,et al.  View-independent Hand Posture Recognition from Single Depth Images Using PCA and Flusser Moments , 2012, 2012 Eighth International Conference on Signal Image Technology and Internet Based Systems.

[15]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..

[16]  Robin R. Murphy,et al.  Hand gesture recognition with depth images: A review , 2012, 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication.

[17]  David W. Murray,et al.  Regression-based Hand Pose Estimation from Multiple Cameras , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[18]  Ulrich Neumann,et al.  Real-time Hand Pose Recognition Using Low-Resolution Depth Images , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[19]  Stan Sclaroff,et al.  Estimating 3D hand pose from a cluttered image , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[20]  Antonis A. Argyros,et al.  Markerless and Efficient 26-DOF Hand Pose Recovery , 2010, ACCV.

[21]  Wannes Meert,et al.  Rule-based Hand Posture Recognition using Qualitative Finger Configurations Acquired with the Kinect , 2013, ICPRAM.