Dynamic Hand Gesture Recognition using Hidden Markov Model by Microsoft Kinect Sensor

Hand gesture recognition is one of the leading applications of human computer interaction. With diversity of applications of hand gesture recognition, sign language interpretation is the most demanding application. In this paper, dynamic hand gesture recognition for few subset of Indian sign language recognition was considered. The use of depth camera such as Kinect sensor gave skeleton information of signer body. After detailed study of dynamic ISL vocabulary with reference to skeleton joint information, angle has identified as a feature with reference to two moving hand. Here, real time video has been captured and gesture was recognized using Hidden Markov Model (HMM). Ten state HMM model was designed and normalized angle feature of dynamic sign was being observed. Maximum likelihood probability symbol was considered as a recognized gesture. Algorithm has been tested on ISL 20 dynamic signs of total 800 training set of four persons and achieved 89.25% average accuracy. General Terms Human Computer Interaction, Pattern Recognition, Machine Learning

[1]  Yi Zhang,et al.  The Research and Application of SURF Algorithm Based on Feature Point Selection Algorithm , 2014 .

[2]  K. Assaleh,et al.  Arabic sign language recognition in user-independent mode , 2007, 2007 International Conference on Intelligent and Advanced Systems.

[3]  Aaron F. Bobick,et al.  Parametric Hidden Markov Models for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  F. Wong,et al.  Hidden Markov Model-Based Gesture Recognition with Overlapping Hand-Head/Hand-Hand Estimated Using Kalman Filter , 2012, 2012 Third International Conference on Intelligent Systems Modelling and Simulation.

[5]  Ling Shao,et al.  Enhanced Computer Vision With Microsoft Kinect Sensor: A Review , 2013, IEEE Transactions on Cybernetics.

[6]  F BobickAaron,et al.  Parametric Hidden Markov Models for Gesture Recognition , 1999 .

[7]  Marcel J. T. Reinders,et al.  Sign Language Recognition by Combining Statistical DTW and Independent Classification , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  A. B. M. Shawkat Ali,et al.  HMM based hand gesture recognition: A review on techniques and approaches , 2009, 2009 2nd IEEE International Conference on Computer Science and Information Technology.

[9]  Dimitris N. Metaxas,et al.  A Framework for Recognizing the Simultaneous Aspects of American Sign Language , 2001, Comput. Vis. Image Underst..

[10]  Archana S. Ghotkar,et al.  VISION BASED MULTI-FEATURE HAND GESTURE RECOGNITION FOR INDIAN SIGN LANGUAGE MANUAL SIGNS , 2016 .

[11]  Ayoub Al-Hamadi,et al.  Hand trajectory-based gesture spotting and recognition using HMM , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[12]  P. Rajesh Kumar,et al.  Video Audio Interface for Recognizing Gestures of Indian Sign Language , 2011 .

[13]  Bodo Rosenhahn,et al.  Real-Time Sign Language Recognition Using a Consumer Depth Camera , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[14]  Jonas Beskow,et al.  A Kinect Corpus of Swedish Sign Language Signs , 2013 .

[15]  Sandy Huang,et al.  Interpreting American Sign Language with Kinect , 2011 .

[16]  Ayoub Al-Hamadi,et al.  A Hidden Markov Model-based continuous gesture recognition system for hand motion trajectory , 2008, 2008 19th International Conference on Pattern Recognition.

[17]  R. Shrivastava,et al.  A hidden Markov model based dynamic hand gesture recognition system using OpenCV , 2013, 2013 3rd IEEE International Advance Computing Conference (IACC).