ANN-Based Hand Gesture Recognition Using Self co-articulated Set of Features

ABSTRACT Hand gesture recognition system can be a particularly challenging task because of the presence of several spatio-temporal variations like hand trembling, self co-articulation within a gesture, etc. during the segmentation stage. In this paper, we focus our attention on self co-articulation problem which has not been addressed before. With the addition of a new feature, i.e. removing the hand movement during pause state along with the velocity features within the gestures, we were able to perform the gesture spotting more smoothly for the isolated gestures. Moreover, a new set of novel features was added in the feature extraction stage: (1) number of self co-articulated stroke; (2) orientation of the self co-articulated strokes; (3) position of the hand; (4) distance between the start and end points of a gesture; and (5) ratio between the longest and shortest distances from centre. These features were fed to neural network classifier. The results of experimentation were carried out with 40 gestures and it showed an average true positive rate of 91.25%.

[1]  Martin T. Hagan,et al.  Neural network design , 1995 .

[2]  R. Oka,et al.  Towards the integration of spontaneous speech and gesture based on spotting method , 1996, 1996 IEEE/SICE/RSJ International Conference on Multisensor Fusion and Integration for Intelligent Systems (Cat. No.96TH8242).

[3]  Sung-Il Chien,et al.  Analysis of 3D Hand Trajectory Gestures Using Stroke-Based Composite Hidden Markov Models , 2001, Applied Intelligence.

[4]  Andrew W. Fitzgibbon,et al.  Direct Least Square Fitting of Ellipses , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Prabin Kumar Bora,et al.  Trajectory Guided Recognition of Hand Gestures having only Global Motions , 2008 .

[6]  Pattie Maes,et al.  WUW - wear Ur world: a wearable gestural interface , 2009, CHI Extended Abstracts.

[7]  Narendra Ahuja,et al.  Extraction of 2D Motion Trajectories and Its Application to Hand Gesture Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Min-Hong Han,et al.  The use of maximum curvature points for the recognition of partially occluded objects , 1990, Pattern Recognit..

[9]  Stan Sclaroff,et al.  Sign Language Spotting with a Threshold Model Based on Conditional Random Fields , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  M.K. Bhuyan,et al.  Co-articulation Detection in Hand Gestures , 2005, TENCON 2005 - 2005 IEEE Region 10 Conference.

[11]  Dan Xu,et al.  Online Dynamic Gesture Recognition for Human Robot Interaction , 2015, J. Intell. Robotic Syst..

[12]  Norman I. Badler,et al.  Synthesis and acquisition of laban movement analysis qualitative parameters for communicative gestures , 2001 .

[13]  David G. Stork,et al.  Pattern Classification , 1973 .

[14]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[15]  Ho-Sub Yoon,et al.  Hand gesture recognition using combined features of location, angle and velocity , 2001, Pattern Recognit..

[16]  Juan Pablo Wachs,et al.  Context-based hand gesture recognition for the operating room , 2014, Pattern Recognit. Lett..

[17]  Jin-Hyung Kim,et al.  An HMM-Based Threshold Model Approach for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Samir I. Shaheen,et al.  Sign language recognition using a combination of new vision based features , 2011, Pattern Recognit. Lett..

[19]  P. K. Bora,et al.  Continuous Hand Gesture Segmentation and Co-articulation Detection , 2006, ICVGIP.

[20]  Moncef Gabbouj,et al.  Content-based object retrieval using maximum curvature points in contour images , 1999, Electronic Imaging.

[21]  Stan Sclaroff,et al.  A Unified Framework for Gesture Recognition and Spatiotemporal Gesture Segmentation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Yuji Iwahori,et al.  A novel set of features for continuous hand gesture recognition , 2014, Journal on Multimodal User Interfaces.

[23]  Karen Das,et al.  Indian Sign Language Recognition Using Eigen Value Weighted Euclidean Distance Based Classification Technique , 2013, ArXiv.

[24]  Trevor Darrell,et al.  Hidden Conditional Random Fields , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Naoto Miura,et al.  Extraction of Finger-Vein Patterns Using Maximum Curvature Points in Image Profiles , 2007, MVA.