3D fingertip and palm tracking in depth image sequences

We present a vision-based approach for robust 3D fingertip and palm tracking on depth images using a single Kinect sensor. First the hand is segmented in the depth images by applying depth and morphological constraints. The palm is located by performing distance transform to the hand contour and tracked with a Kalman filter. The fingertips are detected by combining three depth-based features and tracked with a particle filter over successive frames. Quantitative results on synthetic depth sequences show the proposed scheme can track the fingertips quite accurately. Besides, its capabilities are further demonstrated through a real-life human-computer interaction application.

[1]  Haiying Guan,et al.  Model-based 3D hand posture estimation from a single 2D image , 2002, Image Vis. Comput..

[2]  Junsong Yuan,et al.  Robust hand gesture recognition with kinect sensor , 2011, ACM Multimedia.

[3]  Hamid K. Aghajan,et al.  A Multi-touch Surface Using Multiple Cameras , 2007, ACIVS.

[4]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Junsong Yuan,et al.  Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera , 2011, ACM Multimedia.

[6]  Shao-Yi Chien,et al.  Fast fingertip positioning by combining particle filtering with particle random diffusion , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[7]  Sebastian Thrun,et al.  Real-time identification and localization of body parts from depth images , 2010, 2010 IEEE International Conference on Robotics and Automation.

[8]  Ankit Chaudhary,et al.  Tracking of Fingertips and Centers of Palm Using KINECT , 2011, 2011 Third International Conference on Computational Intelligence, Modelling & Simulation.

[9]  Chih-Cheng Chen,et al.  A combined optimization method for solving the inverse kinematics problems of mechanical manipulators , 1991, IEEE Trans. Robotics Autom..

[10]  Tamim Asfour,et al.  Particle Filter-based Fingertip Tracking with Circular Hough Transform Features , 2011, MVA.

[11]  Yoichi Sato,et al.  Real-time tracking of multiple fingertips and gesture recognition for augmented desk interface systems , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[12]  Salah Bourennane,et al.  Three Dimensional Fingertip Tracking in Stereovision , 2005, ACIVS.