Robust Fingertip Detection in a Complex Environment

Fingertip detection has a broad application in gesture recognition and finger tracking. It is also an important foundation of human-computer interaction systems. However, most algorithms are suitable for simple conditions with low accuracy because the hand is a nonrigid object, and its appearance model is complex. To address the challenging problem of accurately detecting fingertips in a complex environment, we propose a novel and robust fingertip detection algorithm in this paper. Unlike existing methods, our study requires no special device or mark, and users are free to move their hands. Via dense optical flow and a skin filter, we perform complete hand region segmentation in a complex environment. We find the maximum value of the local centroid distance outside the centroid circles and identify fingertips. Our algorithm performs favorably compared with common hand region segmentation and fingertip detection methods. Thorough experimentation proves that our proposed algorithm is effective and robust.

[1]  Antonis A. Argyros,et al.  Markerless and Efficient 26-DOF Hand Pose Recovery , 2010, ACCV.

[2]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Richa Singh,et al.  A Robust Skin Color Based Face Detection Algorithm , 2003 .

[4]  Takashi Nakamura,et al.  Double-Crossing: A New Interaction Technique for Hand Gesture Interfaces , 2008, APCHI.

[5]  Ankit Chaudhary,et al.  Fingertip Detection: A Fast Method with Natural Hand , 2012, ArXiv.

[6]  Joseph J. LaViola,et al.  Poster: Real-time markerless kinect based finger tracking and hand gesture recognition for HCI , 2013, 2013 IEEE Symposium on 3D User Interfaces (3DUI).

[7]  Vasile Gui,et al.  FINGER DETECTION IN VIDEO SEQUENCES USING A NEW SPARSE REPRESENTATION , 2011 .

[8]  Xin Zhang,et al.  A new fingertip detection and tracking algorithm and its application on writing-in-the-air system , 2014, 2014 7th International Congress on Image and Signal Processing.

[9]  Gabriel Pérez,et al.  Explicit image detection using YCbCr space color model as skin detection , 2011 .

[10]  Junsong Yuan,et al.  Robust Part-Based Hand Gesture Recognition Using Kinect Sensor , 2013, IEEE Transactions on Multimedia.

[11]  James M. Rehg,et al.  Statistical Color Models with Application to Skin Detection , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[12]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Junsong Yuan,et al.  Depth camera based hand gesture recognition and its applications in Human-Computer-Interaction , 2011, 2011 8th International Conference on Information, Communications & Signal Processing.

[14]  Daniel Thalmann,et al.  Parsing the Hand in Depth Images , 2014, IEEE Transactions on Multimedia.

[15]  Hasan Sajid,et al.  VSig: Hand-gestured signature recognition and authentication with wearable camera , 2015, 2015 IEEE International Workshop on Information Forensics and Security (WIFS).

[16]  Yun Fu,et al.  Human Age Estimation With Regression on Discriminative Aging Manifold , 2008, IEEE Transactions on Multimedia.

[17]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[18]  James L. Crowley,et al.  Finger Tracking as an Input Device for Augmented Reality , 1995 .

[19]  M. Meribout Video Segmentation for Content-based Coding , 2004 .

[20]  Dieter Schmalstieg,et al.  Finger tracking for interaction in augmented environments , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[21]  Antti Oulasvirta,et al.  Interactive Markerless Articulated Hand Motion Tracking Using RGB and Depth Data , 2013, 2013 IEEE International Conference on Computer Vision.

[22]  Ankit Chaudhary,et al.  Automated switching system for skin pixel segmentation in varied lighting , 2012, 2012 19th International Conference on Mechatronics and Machine Vision in Practice (M2VIP).

[23]  Javier Ruiz Hidalgo,et al.  Real-Time Head and Hand Tracking Based on 2.5D Data , 2012 .

[24]  Manolis I. A. Lourakis,et al.  Binocular Hand Tracking and Reconstruction Based on 2D Shape Matching , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[25]  SeungGwan Lee,et al.  Vision‐Based Finger Action Recognition by Angle Detection and Contour Analysis , 2011 .

[26]  Zhang Xien Interaction Method Based on Data Glove in Virtual Environment , 2005 .

[27]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[28]  Kirit R. Rathod,et al.  A survey on Human Computer Interaction Mechanism Using Finger Tracking , 2014, ArXiv.