Hand posture recognition using Hidden Conditional Random Fields

Body-language understanding is essential to human robot interaction, and hand posture recognition is one of the most important components in a body-language recognition system. The existing hand posture recognition approaches based on robust local features such as SIFT can be invariant to background noise and in-plane rotation. However the ignorance of the relationships among local features is a fundamental issue. The part-based models argue that objects of the same category share the same part-structure which consists of parts and relationships among parts. In this paper, a discriminative part-based model, Hidden Conditional Random Fields (HCRFs), is used to recognize hand postures. Although the existing global locations of features have been used to consider large scale dependency among parts in the HCRFs framework, the results are not invariant to in-plane rotation. New features by the distance to the image center are proposed to encode the global relationship as well as to perform in-plane rotation-invariant recognition. The experimental results demonstrate that the proposed approach is in-plane rotation-invariant and outperforms the approach using AdaBoost with SIFT.

[1]  Andrew McCallum,et al.  An Introduction to Conditional Random Fields for Relational Learning , 2007 .

[2]  John D. Lafferty,et al.  Inducing Features of Random Fields , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Yihong Gong,et al.  Machine Learning for Multimedia Content Analysis (Multimedia Systems and Applications) , 2007 .

[4]  Georgios Tziritas,et al.  Face Detection Using Quantized Skin Color Regions Merging and Wavelet Packet Analysis , 1999, IEEE Trans. Multim..

[5]  Pietro Perona,et al.  Weakly Supervised Scale-Invariant Learning of Models for Visual Recognition , 2007, International Journal of Computer Vision.

[6]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[7]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[8]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[9]  Han-Ming Wu.  Kernel Sliced Inverse Regression with Applications to Classification , 2008 .

[10]  Chieh-Chih Wang,et al.  Hand posture recognition using adaboost with SIFT for human robot interaction , 2007 .

[11]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[12]  Trevor Darrell,et al.  Hidden Conditional Random Fields , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Chieh-Chih Wang,et al.  3D active appearance model for aligning faces in 2D images , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.