A novel finger and hand pose estimation technique for real-time hand gesture recognition

Abstract This paper presents a high-level hand feature extraction method for real-time gesture recognition. Firstly, the fingers are modelled as cylindrical objects due to their parallel edge feature. Then a novel algorithm is proposed to directly extract fingers from salient hand edges. Considering the hand geometrical characteristics, the hand posture is segmented and described based on the finger positions, palm center location and wrist position. A weighted radial projection algorithm with the origin at the wrist position is applied to localize each finger. The developed system can not only extract extensional fingers but also flexional fingers with high accuracy. Furthermore, hand rotation and finger angle variation have no effect on the algorithm performance. The orientation of the gesture can be calculated without the aid of arm direction and it would not be disturbed by the bare arm area. Experiments have been performed to demonstrate that the proposed method can directly extract high-level hand feature and estimate hand poses in real-time.

[1]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..

[2]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  M. S. Sahane,et al.  Visual Interpretation Of Hand Gestures For Human Computer Interaction , 2014 .

[4]  Yoichi Sato,et al.  Real-Time Fingertip Tracking and Gesture Recognition , 2002, IEEE Computer Graphics and Applications.

[5]  Ying Wu,et al.  Capturing natural hand articulation , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[6]  Ying Wu,et al.  3D model-based hand tracking using stochastic direct search method , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[7]  Debanga Raj Neog,et al.  Fingertip Detection for Hand Pose Recognition , 2012 .

[8]  Cheng-Chew Lim,et al.  Hand and face segmentation using motion and color cues in digital image sequences , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[9]  Guijin Wang,et al.  Hand posture recognition in video using multiple cues , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[10]  Ulrich Neumann,et al.  Real-time Hand Pose Recognition Using Low-Resolution Depth Images , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[11]  Hamdy M. Kelash,et al.  Faces and Facial Features Detection in Color Images , 2006, Geometric Modeling and Imaging--New Trends (GMAI'06).

[12]  Nikita Gupta,et al.  Real Time Hand Gesture Recognition for Human Computer Interaction , 2016, 2016 IEEE 6th International Conference on Advanced Computing (IACC).

[13]  Thomas S. Huang,et al.  Vision based hand modeling and tracking for virtual teleconferencing and telecollaboration , 1995, Proceedings of IEEE International Conference on Computer Vision.

[14]  Jae Wook Jeon,et al.  Finger extraction from scene with grayscale morphology and BLOB analysis , 2009, 2008 IEEE International Conference on Robotics and Biomimetics.

[15]  Tsukasa Ogasawara,et al.  Hand pose estimation using multi-viewpoint silhouette images , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[16]  Jianbo Su,et al.  Natural hand posture recognition based on Zernike moments and hierarchical classifier , 2008, 2008 IEEE International Conference on Robotics and Automation.

[17]  Yi Li,et al.  Features extraction from hand images based on new detection operators , 2011, Pattern Recognit..

[18]  Danica Kragic,et al.  Monocular real-time 3D articulated hand pose estimation , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[19]  Yoshiaki Shirai,et al.  Real-time 3D hand posture estimation based on 2D appearance retrieval using monocular camera , 2001, Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems.

[20]  Wen-Kai Tai,et al.  Feature alignment approach for hand posture recognition based on curvature scale space , 2008, Neurocomputing.

[21]  James M. Rehg,et al.  Statistical Color Models with Application to Skin Detection , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[22]  Björn Stenger,et al.  Shape context and chamfer matching in cluttered scenes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[23]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[24]  Rainer Stiefelhagen,et al.  Visual recognition of pointing gestures for human-robot interaction , 2007, Image Vis. Comput..

[25]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[26]  Lale Akarun,et al.  Real time hand pose estimation using depth sensors , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[27]  Lars Bretzner,et al.  Hand gesture recognition using multi-scale colour features, hierarchical models and particle filtering , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[28]  Prabin Kumar Bora,et al.  A robust static hand gesture recognition system using geometry based normalizations and Krawtchouk moments , 2013, Pattern Recognit..

[29]  Cao Chuqing,et al.  Real-time multi-hand posture recognition , 2010, 2010 International Conference On Computer Design and Applications.

[30]  Barbara Mazzarino,et al.  Finger Tracking Methods Using EyesWeb , 2005, Gesture Workshop.

[31]  R. Mas,et al.  Real – Time Hand Tracking and Gesture Recognition for Human-Computer Interaction , 2000 .

[32]  N.D. Georganas,et al.  Real-time Vision-based Hand Gesture Recognition Using Haar-like Features , 2007, 2007 IEEE Instrumentation & Measurement Technology Conference IMTC 2007.

[33]  BebisGeorge,et al.  Vision-based hand pose estimation , 2007 .

[34]  Uwe Kiencke,et al.  Finger Localization and Classification in Images based on Generalized Hough Transform and Probabilistic Models , 2006, 2006 9th International Conference on Control, Automation, Robotics and Vision.

[35]  Taein Lee,et al.  Active contour models , 2005 .

[36]  Jochen Triesch,et al.  A System for Person-Independent Hand Posture Recognition against Complex Backgrounds , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Debanga Raj Neog,et al.  Hand pose recognition using geometric features , 2011, 2011 National Conference on Communications (NCC).

[38]  Yoichi Sato,et al.  Real-time tracking of multiple fingertips and gesture recognition for augmented desk interface systems , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[39]  Nam Ik Cho,et al.  Rotation-invariant face detection using angular projections , 2004 .

[40]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Tommaso Gritti,et al.  Practical study on real-time hand detection , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.