Saliency-guided improvement for hand posture detection and recognition

To detect and recognise hand postures against complex backgrounds, we propose a novel model that is constructed by the integration of image saliency and skin information. Although a skin model is a simple and efficient strategy by which to locate skin regions within images, it is easily affected by complex backgrounds, e.g. skin-like background regions and various lighting conditions. To solve this problem, we propose a general image saliency detection method that is then integrated with skin information to improve the performance of hand posture detection. Lastly, a linear Support Vector Machine (SVM) is adopted to recognise hand postures according to the results of hand posture detection. In the experiment, we tested the performance of the proposed image saliency detection method over seven state-of-the-art methods. The saliency-based hand posture detection and recognition model was also evaluated. These experiments show that the proposed model has stable performance for a wide range of images.

[1]  F. Florez,et al.  Hand gesture recognition following the dynamics of a topology-preserving network , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[2]  J. P. Lewis,et al.  SmartCanvas: a gesture-driven intelligent drawing desk system , 2005, IUI.

[3]  Jochen Triesch,et al.  A System for Person-Independent Hand Posture Recognition against Complex Backgrounds , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Paul A. Viola,et al.  Robust Real-time Object Detection , 2001 .

[5]  Ai Poh Loh,et al.  Attention Based Detection and Recognition of Hand Postures Against Complex Backgrounds , 2012, International Journal of Computer Vision.

[6]  Alexander Zelinsky,et al.  Visual gesture interfaces for virtual environments , 2002, Interact. Comput..

[7]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Rainer Stiefelhagen,et al.  Quaternion-Based Spectral Saliency Detection for Eye Fixation Prediction , 2012, ECCV.

[9]  Ying Wu,et al.  A unified approach to salient object detection via low rank matrix recovery , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[11]  Wen-Kai Tai,et al.  Feature alignment approach for hand posture recognition based on curvature scale space , 2008, Neurocomputing.

[12]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Nicu Sebe,et al.  Image saliency by isocentric curvedness and color , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[14]  ELENA SÁNCHEZ-NIELSEN,et al.  Visual Interaction through Hand Gesture Recognition using Hausdorff Matching , 2003 .

[15]  Rainer Stiefelhagen,et al.  Multimodal saliency-based attention for object-based scene analysis , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Thomas Serre,et al.  Object recognition with features inspired by visual cortex , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[17]  Dieter W. Fellner,et al.  Interaction with hand gesture for a back-projection wall , 2004, Proceedings Computer Graphics International, 2004..

[18]  T. Duckett,et al.  VOCUS : A Visual Attention System for Object Detection and Goal-directed Search , 2010 .

[19]  Ying Wu,et al.  View-independent recognition of hand postures , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[20]  HongJiang Zhang,et al.  Contrast-based image attention analysis by using fuzzy growing , 2003, MULTIMEDIA '03.

[21]  Richard Bowden,et al.  A boosted classifier tree for hand shape detection , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[22]  Sabine Süsstrunk,et al.  Salient Region Detection and Segmentation , 2008, ICVS.

[23]  Ying Wu,et al.  3D model-based visual hand tracking , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[24]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[25]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[26]  Rainer Stiefelhagen,et al.  How the distribution of salient objects in images influences salient object detection , 2013, 2013 IEEE International Conference on Image Processing.

[27]  Chin-Chen Chang,et al.  Adaptive multiple sets of CSS features for hand posture recognition , 2006, Neurocomputing.

[28]  Christoph H. Lampert,et al.  Beyond sliding windows: Object localization by efficient subwindow search , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Simone Frintrop,et al.  A Real-time Visual Attention System Using Integral Images , 2007, ICVS 2007.

[30]  Armin B. Cremers,et al.  Boosting scalable gradient features for adaptive real-time tracking , 2011, 2011 IEEE International Conference on Robotics and Automation.

[31]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[32]  Mathias Kölsch,et al.  Robust hand detection , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[33]  Rainer Stiefelhagen,et al.  Predicting human gaze using quaternion DCT image signature saliency and face detection , 2012, 2012 IEEE Workshop on the Applications of Computer Vision (WACV).

[34]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[35]  Ali Borji,et al.  Boosting bottom-up and top-down visual features for saliency estimation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Lars Bretzner,et al.  Hand gesture recognition using multi-scale colour features, hierarchical models and particle filtering , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[37]  M.K. Bhuyan,et al.  Threshold Finite State Machine for Vision Based Gesture Recognition , 2005, 2005 Annual IEEE India Conference - Indicon.