Real-time fingertip detection based on depth data

In this paper we propose a novel method to detect fingertip using depth data. The first step of our method is to segment hand from depth map precisely. Then a two layer hand model is constructed to detect self-occlusion and mitigate its impact. In the next step an extended graph model of hand is built to locate and label finger bases. Then we generate heat maps of finger bases to detect finger regions even fingers are closed or adhesion occurs. Finally fingertips are located on fingers by geodesic paths. Experiments on different finger motions and hand rotations show that our framework performs accurately when hand pose and rotation change. Compared with other approaches our method shows less errors and robust to depth noise.

[1]  Yonghong Song,et al.  Real Time Fingertip Detection with Kinect Depth Image Sequences , 2014, 2014 22nd International Conference on Pattern Recognition.

[2]  Junsong Yuan,et al.  Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera , 2011, ACM Multimedia.

[3]  Ankit Chaudhary,et al.  Tracking of Fingertips and Centers of Palm Using KINECT , 2011, 2011 Third International Conference on Computational Intelligence, Modelling & Simulation.

[4]  David W. Murray,et al.  Regression-based Hand Pose Estimation from Multiple Cameras , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[5]  Li Cheng,et al.  Efficient Hand Pose Estimation from a Single Depth Image , 2013, 2013 IEEE International Conference on Computer Vision.

[6]  Yoichi Sato,et al.  Real-Time Fingertip Tracking and Gesture Recognition , 2002, IEEE Computer Graphics and Applications.

[7]  Stan Sclaroff,et al.  Estimating 3D hand pose from a cluttered image , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[8]  Daniel Thalmann,et al.  3D fingertip and palm tracking in depth image sequences , 2012, ACM Multimedia.

[9]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[10]  Cheng-Chang Lien A scalable model-based hand posture analysis system , 2004, Machine Vision and Applications.

[11]  Yi Li,et al.  Hand gesture recognition using Kinect , 2012, 2012 IEEE International Conference on Computer Science and Automation Engineering.

[12]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[13]  Tae-Kyun Kim,et al.  Real-Time Articulated Hand Pose Estimation Using Semi-supervised Transductive Regression Forests , 2013, 2013 IEEE International Conference on Computer Vision.

[14]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..