Real-time hand tracking using synergistic inverse kinematics

We present a method for real-time bare hand tracking that utilizes natural hand synergies to reduce the complexity and improve the plausibility of the hand posture estimation. The hand pose and posture are estimated by fitting a virtual hand model to the 3D point cloud obtained from a Kinect camera using an inverse kinematics approach. We use real human hand movements captured with a Vicon motion tracking system as the ground truth for deriving natural hand synergies based on principal component analysis. These synergies are integrated in the tracking scheme by optimizing the posture in a reduced parameter space. Tracking in this reduced space combined with joint limit avoidance constrains the posture estimation to natural hand articulations. The information loss associated with dimension reduction can be dealt with by employing a hierarchical optimization scheme. We show that our synergistic hand tracking approach improves runtime performance and increases the quality of the posture estimation.

[1]  Luc Van Gool,et al.  Tracking a hand manipulating an object , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[2]  J. Foley.  The co-ordination and regulation of movements , 1968 .

[3]  Ashley Reid,et al.  A comparison of linear skinning techniques for character animation , 2007, AFRIGRAPH '07.

[4]  S. Buss.  Introduction to Inverse Kinematics with Jacobian Transpose , Pseudoinverse and Damped Least Squares methods , 2004 .

[5]  Helge J. Ritter,et al.  Real-time hand tracking with a color glove for the actuation of anthropomorphic robot hands , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[6]  J. F. Soechting,et al.  Postural Hand Synergies for Tool Use , 1998, The Journal of Neuroscience.

[7]  Andreas Daffertshofer,et al.  PCA in studying coordination and variability: a tutorial. , 2004, Clinical biomechanics.

[8]  Antonis A. Argyros,et al.  Tracking the articulated motion of two strongly interacting hands , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Jovan Popović,et al.  Real-time hand-tracking with a color glove , 2009, SIGGRAPH 2009.

[10]  Antonis A. Argyros,et al.  Full DOF tracking of a hand interacting with an object by modeling occlusions and physical constraints , 2011, 2011 International Conference on Computer Vision.

[11]  Antonis A. Argyros,et al.  Efficient model-based 3D tracking of hand articulations using Kinect , 2011, BMVC.

[12]  Aaron Hertzmann,et al.  Style-based inverse kinematics , 2004, ACM Trans. Graph..

[13]  Samuel R. Buss,et al.  Selectively Damped Least Squares for Inverse Kinematics , 2005, J. Graph. Tools.

[14]  Aaron Hertzmann,et al.  Style-based inverse kinematics , 2004, SIGGRAPH 2004.

[15]  Tosiyasu L. Kunii,et al.  Model-based analysis of hand posture , 1995, IEEE Computer Graphics and Applications.

[16]  Ying Wu,et al.  Capturing natural hand articulation , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[17]  Helge J. Ritter,et al.  Robust tracking of human hand postures for robot teaching , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Luc Van Gool,et al.  Smart particle filtering for high-dimensional tracking , 2007, Comput. Vis. Image Underst..

[19]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[20]  Helge Ritter,et al.  Analysis of Hand Synergies for Inverse Kinematics Hand Tracking , 2013 .

[21]  Danica Kragic,et al.  Hands in action: real-time 3D reconstruction of hands in interaction with objects , 2010, 2010 IEEE International Conference on Robotics and Automation.

[22]  Luc Van Gool,et al.  Motion Capture of Hands in Action Using Discriminative Salient Points , 2012, ECCV.

[23]  Rajiv V. Dubey,et al.  A weighted least-norm solution based scheme for avoiding joint limits for redundant joint manipulators , 1993, IEEE Trans. Robotics Autom..

[24]  Qionghai Dai,et al.  Video-based hand manipulation capture through composite motion control , 2013, ACM Trans. Graph..

[25]  Yen-Wei Chen,et al.  Articulated Hand Motion Tracking Using ICA-based Motion Analysis and Particle Filtering , 2006, J. Multim..

[26]  Jessica K. Hodgins,et al.  Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces , 2004, SIGGRAPH 2004.

[27]  Andrea d'Avella,et al.  Matrix factorization algorithms for the identification of muscle synergies: evaluation on simulated and experimental data sets. , 2006, Journal of neurophysiology.

[28]  Helge J. Ritter,et al.  Approaching Manual Intelligence , 2010, KI - Künstliche Intelligenz.