A hybrid method using haar-like and skin-color algorithm for hand posture detection, recognition and tracking

Human hand posture detection and recognition is a challenging problem in computer vision. We introduce an algorithm that is capable to recognize hand posture in a sophisticated background. The system combines two algorithms to achieve better detection rate for hand. Recently Viola et al. in [10] have introduced a rapid object detection scheme; we use this approach to detect the hand posture in the first set of consecutive frames. The chromatic color distribution of skin can be found within this cluster. As the shape of hand posture keep changing in the subsequent frames, the skin regions updated dynamically. The classification of hand posture makes use of static feature for locating and counting hand fingers. Kalman Filter is used to track the face and hand blobs based on their position. In the experiments, we have tested our system in various environments, and results showed effectiveness of the approach.

[1]  Paul A. Viola,et al.  Robust Real-time Object Detection , 2001 .

[2]  Karl-Friedrich Kraiss,et al.  Extraction of 3D hand shape and posture from image sequences for sign language recognition , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[3]  Mathias Kölsch,et al.  Robust hand detection , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[4]  D.V. Thombre,et al.  Human detection and tracking using image segmentation and Kalman filter , 2009, 2009 International Conference on Intelligent Agent & Multi-Agent Systems.

[5]  Yoshiaki Shirai,et al.  Hand shape estimation under complex backgrounds for sign language recognition , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[6]  Richard Bowden,et al.  A boosted classifier tree for hand shape detection , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[7]  Tosiyasu L. Kunii,et al.  Model-based analysis of hand posture , 1995, IEEE Computer Graphics and Applications.

[8]  Matthew Turk,et al.  View-based interpretation of real-time optical flow for gesture recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[9]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[10]  Mohinder S. Grewal,et al.  Kalman Filtering: Theory and Practice , 1993 .

[11]  Niels da Vitoria Lobo,et al.  Open Hand Detection in a Cluttered Single Image using Finger Primitives , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[12]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[13]  Stan Sclaroff,et al.  Automatic 2D Hand Tracking in Video Sequences , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[14]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[15]  S. Sclaroff,et al.  Hand Pose Reconstruction Using Specialized Mappings , 2000 .

[16]  Yoshiaki Shirai,et al.  Hand gesture estimation and model refinement using monocular camera-ambiguity limitation by inequality constraints , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[17]  Alexander H. Waibel,et al.  A real-time face tracker , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[18]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Roland T. Chin,et al.  On image analysis by the methods of moments , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Shan Lu,et al.  Color-based hands tracking system for sign language recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[21]  Yoav Freund,et al.  A Short Introduction to Boosting , 1999 .

[22]  Nitin V. Pujari,et al.  Finger Detection for Sign Language Recognition , 2009 .

[23]  Tomaso A. Poggio,et al.  A general framework for object detection , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[24]  Tae-Woong Yoo,et al.  A fast algorithm for tracking human faces based on chromatic histograms , 1999, Pattern Recognit. Lett..

[25]  Michal Kawulok.  Dynamic Skin Detection in Color Images for Sign Language Recognition , 2008, ICISP.

[26]  Rómer Rosales,et al.  3D Hand Pose Reconstruction Using Specialized Mappings , 2001, ICCV.

[27]  Yoshiaki Shirai,et al.  Extraction of Hand Features for Recognition of Sign Language Words , 2002 .

[28]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[29]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Sung Kwan Kang,et al.  Color Based Hand and Finger Detection Technology for User Interaction , 2008, 2008 International Conference on Convergence and Hybrid Information Technology.

[31]  Lars Bretzner,et al.  Hand gesture recognition using multi-scale colour features, hierarchical models and particle filtering , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[32]  Ying Wu,et al.  Hand modeling, analysis and recognition , 2001, IEEE Signal Process. Mag..

[33]  Ho-Sub Yoon,et al.  Hand gesture recognition using combined features of location, angle and velocity , 2001, Pattern Recognit..