Movement Epenthesis Detection for Continuous Sign Language Recognition

Abstract Automatic sign language recognition (SLR) is a current area of research as this is meant to serve as a substitute for sign language interpreters. In this paper, we present the design of a continuous SLR system that can extract out the meaningful signs and consequently recognize them. Here, we have used height of the hand trajectory as a salient feature for separating out the meaningful signs from the movement epenthesis patterns. Further, we have incorporated a unique set of spatial and temporal features for efficient recognition of the signs encapsulated within the continuous sequence. The implementation of an efficient hand segmentation and hand tracking technique makes our system robust to complex background as well as background with multiple signers. Experiments have established that our proposed system can identify signs from a continuous sign stream with a 92.8% spotting rate.

[1]  Annelies Braffort,et al.  Toward Modeling Sign Language Coarticulation , 2009, Gesture Workshop.

[2]  Daniel Kelly,et al.  Recognizing Spatiotemporal Gestures and Movement Epenthesis in Sign Language , 2009, 2009 13th International Machine Vision and Image Processing Conference.

[3]  Ruiduo Yang,et al.  Handling Movement Epenthesis and Hand Segmentation Ambiguities in Continuous Sign Language Recognition Using Nested Dynamic Programming , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Onno Crasborn,et al.  Coarticulation of hand height in Sign Language of the Netherlands is affected by contact type , 2013, J. Phonetics.

[5]  Kandarpa Kumar Sarma,et al.  A Conditional Random Field Based Indian Sign Language Recognition System under Complex Background , 2014, 2014 Fourth International Conference on Communication Systems and Network Technologies.

[6]  Joseph N. Wilson,et al.  Handbook of computer vision algorithms in image algebra , 1996 .

[7]  Kandarpa Kumar Sarma,et al.  A novel hand segmentation method for multiple-hand gesture recognition system under complex background , 2014, 2014 International Conference on Signal Processing and Integrated Networks (SPIN).

[8]  Abdesselam Bouzerdoum,et al.  Skin segmentation using color pixel classification: analysis and comparison , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  W.-S. Chen,et al.  Movement Epenthesis Generation Using NURBS-Based Spatial Interpolation , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Adrian Kaehler,et al.  Learning opencv, 1st edition , 2008 .

[11]  Ruiduo Yang,et al.  Detecting Coarticulation in Sign Language using Conditional Random Fields , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[12]  Stan Sclaroff,et al.  Sign Language Spotting with a Threshold Model Based on Conditional Random Fields , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  M.K. Bhuyan,et al.  Co-articulation Detection in Hand Gestures , 2005, TENCON 2005 - 2005 IEEE Region 10 Conference.

[14]  Qing Chen,et al.  Hand Gesture Recognition Using Haar-Like Features and a Stochastic Context-Free Grammar , 2008, IEEE Transactions on Instrumentation and Measurement.