A depth-based approach for 3D dynamic gesture recognition

In this paper we propose a recognition technique of 3D dynamic gesture for human robot interaction (HRI) based on depth information provided by Kinect sensor. The body is tracked using the skeleton algorithm provided by the Kinect SDK. The main idea of this work is to compute the angles of the upper body joints which are active when executing gesture. The variation of these angles are used as inputs of Hidden Markov Models (HMM) in order to recognize the dynamic gestures. Results demonstrate the robustness of our method against environmental conditions such as illumination changes and scene complexity due to using depth information only.

[1]  Fakhreddine Ababsa.  Robust Extended Kalman Filtering for camera pose tracking using 2D to 3D lines correspondences , 2009, 2009 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.

[2]  Yoshua Bengio,et al.  Input-output HMMs for sequence processing , 1996, IEEE Trans. Neural Networks.

[3]  Ming Li,et al.  Viewing Sea Level by a One-Dimensional Random Function with Long Memory , 2011 .

[4]  Jen-Tzung Chien,et al.  Bayesian Sensing Hidden Markov Models , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[5]  Zhengyou Zhang,et al.  Microsoft Kinect Sensor and Its Effect , 2012, IEEE Multim..

[6]  Qing Chen,et al.  Hand Gesture Recognition Using Haar-Like Features and a Stochastic Context-Free Grammar , 2008, IEEE Transactions on Instrumentation and Measurement.

[7]  Dan Xu,et al.  Integrated approach of skin-color detection and depth information for hand and face localization , 2011, 2011 IEEE International Conference on Robotics and Biomimetics.

[8]  Xiaoyan Wang,et al.  Hidden-Markov-Models-Based Dynamic Hand Gesture Recognition , 2012 .

[9]  S. E. Ghobadi,et al.  Hand Segmentation Using 2D/3D Images , 2007 .

[10]  Siddharth Swarup Rautaray,et al.  A Real Time Hand Tracking System for Interactive Applications , 2011 .

[11]  Aggelos K. Katsaggelos,et al.  Part-based initialization for hand tracking , 2010, 2010 IEEE International Conference on Image Processing.

[12]  Luke S. Zettlemoyer,et al.  Learning from Unscripted Deictic Gesture and Language for Human-Robot Interactions , 2014, AAAI.

[13]  A. Corradini,et al.  Dynamic time warping for off-line recognition of a small gesture vocabulary , 2001, Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems.

[14]  Sebastian Thrun,et al.  Real-time identification and localization of body parts from depth images , 2010, 2010 IEEE International Conference on Robotics and Automation.

[15]  Malik Mallem,et al.  Robust line tracking using a particle filter for camera pose estimation , 2006, VRST '06.

[16]  Dan Xu,et al.  Online Dynamic Gesture Recognition for Human Robot Interaction , 2015, J. Intell. Robotic Syst..

[17]  Ayoub Al-Hamadi,et al.  A Hidden Markov Model-Based Isolated and Meaningful Hand Gesture Recognition , 2008 .

[18]  Weihua Sheng,et al.  Human gesture recognition through a Kinect sensor , 2012, 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[19]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[20]  Ayoub Al-Hamadi,et al.  Real-Time Capable System for Hand Gesture Recognition Using Hidden Markov Models in Stereo Color Image Sequences , 2008, J. WSCG.

[21]  Gerhard Rigoll,et al.  Hidden Markov model based continuous online gesture recognition , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[22]  Toshiaki Ejima,et al.  Real-Time hand Gesture Recognition Using Pseudo 3-D Hidden Markov Model , 2006, 2006 5th IEEE International Conference on Cognitive Informatics.

[23]  Peter Tino,et al.  IEEE Transactions on Neural Networks , 2009 .

[24]  Yiping Yang,et al.  Real-time Hand Gesture Recognition from Depth Images Using Convex Shape Decomposition Method , 2014, J. Signal Process. Syst..

[25]  Søren Hauberg,et al.  Unscented Kalman Filtering for Articulated Human Tracking , 2011, SCIA.

[26]  Michael Werman,et al.  Fusing Time-of-Flight Depth and Color for Real-Time Segmentation and Tracking , 2009, Dyn3D.

[27]  Jake K. Aggarwal,et al.  Human detection using depth information by Kinect , 2011, CVPR 2011 WORKSHOPS.

[28]  Trevor Darrell,et al.  Hidden Conditional Random Fields for Gesture Recognition , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).