Real-Time Hand Posture Recognition for Human-Robot Interaction Tasks

In this work, we present a multiclass hand posture classifier useful for human-robot interaction tasks. The proposed system is based exclusively on visual sensors, and it achieves a real-time performance, whilst detecting and recognizing an alphabet of four hand postures. The proposed approach is based on the real-time deformable detector, a boosting trained classifier. We describe a methodology to design the ensemble of real-time deformable detectors (one for each hand posture that can be classified). Given the lack of standard procedures for performance evaluation, we also propose the use of full image evaluation for this purpose. Such an evaluation methodology provides us with a more realistic estimation of the performance of the method. We have measured the performance of the proposed system and compared it to the one obtained by using only the sampled window approach. We present detailed results of such tests using a benchmark dataset. Our results show that the system can operate in real time at about a 10-fps frame rate.

[1]  Sébastien Marcel,et al.  Hand gesture recognition using input-output hidden Markov models , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[2]  Pascal Fua,et al.  A Real-Time Deformable Detector , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..

[4]  P. M. Mendes,et al.  FBG Sensing Glove for Monitoring Hand Posture , 2011, IEEE Sensors Journal.

[5]  Frédéric Lerasle,et al.  Two-handed gesture recognition and fusion with speech to command a robot , 2011, Autonomous Robots.

[6]  Haitham Hasan,et al.  Retraction Note to: Human–computer interaction using vision-based hand gesture recognition systems: a survey , 2017, Neural Computing and Applications.

[7]  Manolis I. A. Lourakis,et al.  Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse , 2006, ECCV Workshop on HCI.

[8]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[9]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Pietro Perona,et al.  Pedestrian Detection: An Evaluation of the State of the Art , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[12]  Soowoong Kim,et al.  Vision-based cleaning area control for cleaning robots , 2012, IEEE Transactions on Consumer Electronics.

[13]  Qing Chen,et al.  Hand Gesture Recognition Using Haar-Like Features and a Stochastic Context-Free Grammar , 2008, IEEE Transactions on Instrumentation and Measurement.

[14]  Akinori Sasaki,et al.  Hand posture estimation using 3D range data for an evaluation system of human hand manipulation , 2011, IECON 2011 - 37th Annual Conference of the IEEE Industrial Electronics Society.

[15]  Tae-Kyun Kim,et al.  Canonical Correlation Analysis of Video Volume Tensors for Action Categorization and Detection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[17]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Rini Akmeliawati,et al.  A hybrid method using haar-like and skin-color algorithm for hand posture detection, recognition and tracking , 2010, 2010 IEEE International Conference on Mechatronics and Automation.

[19]  Jochen J. Steil,et al.  Human-robot interaction for learning and adaptation of object movements , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Luc Van Gool,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[21]  Jakub Nalepa,et al.  Spatial-based skin detection using discriminative skin-presence features , 2014, Pattern Recognit. Lett..

[22]  Ling Shao,et al.  Learning Discriminative Representations from RGB-D Video Data , 2013, IJCAI.

[23]  Jochen Triesch,et al.  A System for Person-Independent Hand Posture Recognition against Complex Backgrounds , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Juan Pablo Wachs,et al.  HEGM: A hierarchical elastic graph matching for hand gesture recognition , 2014, Pattern Recognit..

[25]  Ai Poh Loh,et al.  Hand Posture and Face Recognition Using a Fuzzy-Rough Approach , 2010, Int. J. Humanoid Robotics.

[26]  Thi-Thanh-Hai Tran,et al.  Invariant lighting hand posture classification , 2010, 2010 IEEE International Conference on Progress in Informatics and Computing.

[27]  Antonis A. Argyros,et al.  Scale invariant and deformation tolerant partial shape matching , 2011, Image Vis. Comput..

[28]  Riccardo Leonardi,et al.  XKin: an open source framework for hand pose and gesture recognition using kinect , 2014, The Visual Computer.

[29]  Liwei Liu,et al.  Hand posture recognition using finger geometric feature , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[30]  Ai Poh Loh,et al.  Attention Based Detection and Recognition of Hand Postures Against Complex Backgrounds , 2012, International Journal of Computer Vision.

[31]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[32]  H. S. Wolff,et al.  iRun: Horizontal and Vertical Shape of a Region-Based Graph Compression , 2022, Sensors.

[33]  D. Geman,et al.  Stationary Features and Cat Detection , 2008 .

[34]  Yi-Qing Wang,et al.  An Analysis of the Viola-Jones Face Detection Algorithm , 2014, Image Process. Line.

[35]  Nanning Zheng,et al.  Training more discriminative multi-class classifiers for hand detection , 2015, Pattern Recognit..

[36]  Yue Wang,et al.  Real-time hand posture recognition based on hand dominant line using kinect , 2013, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[37]  Chuqing Cao,et al.  Real-Time Hand Posture Recognition Using Haar-Like and Topological Feature , 2010, 2010 International Conference on Machine Vision and Human-machine Interface.

[38]  Prahlad Vadakkepat,et al.  HAND POSTURE AND FACE RECOGNITION USING A FUZZY-ROUGH APPROACH , 2010 .

[39]  Alexei A. Efros,et al.  Ensemble of exemplar-SVMs for object detection and beyond , 2011, 2011 International Conference on Computer Vision.

[40]  Yi Yao,et al.  Hand Posture Recognition Using SURF with Adaptive Boosting , 2012, BMVC 2012.

[41]  Chieh-Chih Wang,et al.  Hand posture recognition using adaboost with SIFT for human robot interaction , 2007 .

[42]  Junsong Yuan,et al.  Robust Part-Based Hand Gesture Recognition Using Kinect Sensor , 2013, IEEE Transactions on Multimedia.

[43]  Hedvig Kjellström,et al.  Inferring hand pose: A comparative study of visual shape features , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[44]  Jiri Matas,et al.  Weighted Sampling for Large-Scale Boosting , 2008, BMVC.

[45]  R. S. Jadon,et al.  A REVIEW OF VISION BASED HAND GESTURES RECOGNITION , 2009 .

[46]  Luigi Gallo,et al.  View-independent Hand Posture Recognition from Single Depth Images Using PCA and Flusser Moments , 2012, 2012 Eighth International Conference on Signal Image Technology and Internet Based Systems.

[47]  Tae-Kyun Kim,et al.  Tensor Canonical Correlation Analysis for Action Classification , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Haitao Yu,et al.  Real-Time Markerless Hand Gesture Recognition with Depth Camera , 2012, PCM.

[49]  Haitham Hasan,et al.  RETRACTED ARTICLE: Human–computer interaction using vision-based hand gesture recognition systems: a survey , 2013, Neural Computing and Applications.