An Accurate Algorithm for the Identification of Fingertips Using an RGB-D Camera

RGB-D cameras and depth sensors have made possible the development of an uncountable number of applications in the field of human-computer interactions. Such applications, varying from gaming to medical, have made possible because of the capability of such sensors of elaborating depth maps of the placed ambient. In this context, aiming to realize a sound basis for future applications relevant to the movement and to the pose of hands, we propose a new approach to recognize fingertips and to identify their position by means of the Microsoft Kinect technology. The experimental results exhibit a really good identification rate, an execution speed faster than the frame rate with no meaningful latencies, thus allowing the use of the proposed system in real time applications. Furthermore, the scored identification accuracy confirms the excellent capability of following also little movements of the hand and it encourages the real possibility of successive implementations in more complex gesture recognition systems.

[1]  Wolfgang Hürst,et al.  Gesture-based interaction via finger tracking for mobile augmented reality , 2011, Multimedia Tools and Applications.

[2]  Rod McCall,et al.  Lightweight palm and finger tracking for real-time 3D gesture control , 2011, 2011 IEEE Virtual Reality Conference.

[3]  Robert W. Lindeman,et al.  A multi-class pattern recognition system for practical finger spelling translation , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[4]  Stan Sclaroff,et al.  A Unified Framework for Gesture Recognition and Spatiotemporal Gesture Segmentation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Domenico Prattichizzo,et al.  Using Kinect for hand tracking and rendering in wearable haptics , 2011, 2011 IEEE World Haptics Conference.

[6]  Murray Griffin,et al.  Activity-promoting gaming systems in exercise and rehabilitation. , 2011, Journal of rehabilitation research and development.

[7]  Janne Heikkilä,et al.  Adaptive Motion-Based Gesture Recognition Interface for Mobile Phones , 2008, ICVS.

[8]  Haiying Guan,et al.  3D hand pose retrieval from a single 2D image , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[9]  Albert A. Rizzo,et al.  Development and evaluation of low cost game-based balance rehabilitation tool using the microsoft kinect sensor , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[10]  Massimo Panella,et al.  Neural networks with quantum architecture and quantum learning , 2011, Int. J. Circuit Theory Appl..

[11]  Tobias Höllerer,et al.  Handy AR: Markerless Inspection of Augmented Reality Objects Using Fingertip Tracking , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[12]  Jorge Lobo,et al.  Hand Gesture Recognition Using Color and Depth Images Enhanced with Hand Angular Pose Data * , 2022 .

[13]  Bernd Fröhlich,et al.  Finger and hand detection for multi-touch interfaces based on maximally stable extremal regions , 2012, ITS '12.

[14]  Chih-Chang Yu,et al.  Vision-Based Fingertip-Writing Character Recognition , 2011, J. Signal Process. Syst..

[15]  Thad Starner,et al.  A novel approach to American Sign Language (ASL) phrase verification using reversed signing , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[16]  M. Teague Image analysis via the general theory of moments , 1980 .

[17]  SeungGwan Lee,et al.  Vision‐Based Finger Action Recognition by Angle Detection and Contour Analysis , 2011 .

[18]  Yang Li,et al.  A real-time multi-cue hand tracking algorithm based on computer vision , 2010, 2010 IEEE Virtual Reality Conference (VR).

[19]  Björn Stenger,et al.  Filtering using a tree-based estimator , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[20]  Massimo Panella,et al.  An RNS Architecture for Quasi-Chaotic Oscillators , 2003, J. VLSI Signal Process..

[21]  Yi Li,et al.  Hand gesture recognition using Kinect , 2012, 2012 IEEE International Conference on Computer Science and Automation Engineering.

[22]  Marjorie Skubic,et al.  Evaluation of an inexpensive depth camera for passive in-home fall risk assessment , 2011, 2011 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[23]  Ankit Chaudhary,et al.  Tracking of Fingertips and Centers of Palm Using KINECT , 2011, 2011 Third International Conference on Computational Intelligence, Modelling & Simulation.

[24]  Lale Akarun,et al.  Real time hand pose estimation using depth sensors , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[25]  Kwang-Seok Hong,et al.  Finger Gesture Estimation for Mobile Device User Interface Using a Rear-Facing Camera , 2011 .

[26]  Raffaele Parisi,et al.  Source Localization in Reverberant Environments by Consistent Peak Selection , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[27]  Xin Zhang,et al.  Real-time fingertip tracking and detection using Kinect depth sensor for a new writing-in-the air system , 2012, ICIMCS '12.

[28]  Daniel Thalmann,et al.  3D fingertip and palm tracking in depth image sequences , 2012, ACM Multimedia.

[29]  Yan Wen,et al.  A robust method of detecting hand gestures using depth sensors , 2012, 2012 IEEE International Workshop on Haptic Audio Visual Environments and Games (HAVE 2012) Proceedings.

[30]  Linda Denehy,et al.  Validity of the Microsoft Kinect for assessment of postural control. , 2012, Gait & posture.

[31]  周炯,et al.  Kind of static method and system for object reconstruction , 2014 .

[32]  Ankit Chaudhary,et al.  A Vision based Geometrical Method to find Fingers Positions in Real Time Hand Gesture Recognition , 2012, J. Softw..

[33]  Lianwen Jin,et al.  A Novel Vision-Based Finger-Writing Character Recognition System , 2007, J. Circuits Syst. Comput..

[34]  Jack Sklansky,et al.  Measuring Concavity on a Rectangular Mosaic , 1972, IEEE Transactions on Computers.

[35]  Ankit Chaudhary,et al.  A Survey on Hand Gesture Recognition in Context of Soft Computing , 2011 .

[36]  Massimo Panella,et al.  The quantum approach leading from evolutionary to exhaustive optimization , 2012 .

[37]  Yoichi Sato,et al.  Real-Time Fingertip Tracking and Gesture Recognition , 2002, IEEE Computer Graphics and Applications.

[38]  Tamim Asfour,et al.  Particle Filter-based Fingertip Tracking with Circular Hough Transform Features , 2011, MVA.

[39]  R. F. Sari,et al.  Color recognition system with augmented reality concept and finger interaction: Case study for color blind aid system , 2012, 2011 Ninth International Conference on ICT and Knowledge Engineering.

[40]  Yun Liao,et al.  Fingertips Detection Algorithm Based on Skin Colour Filtering and Distance Transformation , 2012, 2012 12th International Conference on Quality Software.

[41]  Richard E. Ladner,et al.  Input finger detection for nonvisual touch screen text entry in Perkinput , 2012, Graphics Interface.

[42]  Patrick van der Smagt,et al.  Patient performance evaluation using Kinect and Monte Carlo-based finger tracking , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[43]  Yao-Jen Chang,et al.  A Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. , 2011, Research in developmental disabilities.