Appearance-based Arabic Sign Language recognition using Hidden Markov Models

In this paper, we propose a new method to solve sign language recognition problem using appearance-based features. Particularly, Local Binary Patterns (LBP) are employed to describe the texture and the shape of sign language images. The feature vector resulted from LBP operator is further reduced using Principal Component Analysis (PCA). The appearance-based features are classified using Hidden Markov Models (HMM). The performance of the proposed method is measured using Arabic Sign Language (ArSL) database. The proposed method does not rely on the use of data gloves or other means of input devices, and it allows the deaf signers to perform gestures without imposing any restriction on clothing or image background. Using LBP and PCA features, a recognition rate up to 99.97% was achieved for signer dependent recognition.

[1]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Khaled Assaleh,et al.  Telescopic Vector Composition and Polar Accumulated Motion Residuals for Feature Extraction in Arabic Sign Language Recognition , 2007, EURASIP J. Image Video Process..

[3]  Omar M. Al-Jarrah,et al.  Recognition of gestures in Arabic sign language using neuro-fuzzy systems , 2001, Artif. Intell..

[4]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[5]  J. Trojanová,et al.  Local Binary Pattern based features for sign language recognition , 2012, Pattern Recognition and Image Analysis.

[6]  Khaled Assaleh,et al.  Video-based signer-independent Arabic sign language recognition using hidden Markov models , 2009, Appl. Soft Comput..

[7]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Hermann Ney,et al.  Spoken language processing techniques for sign language recognition and translation , 2008, Technology and Disability.

[9]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[10]  Hermann Hienz,et al.  HMM-Based Continuous Sign Language Recognition Using Stochastic Grammars , 1999, Gesture Workshop.

[11]  Mohammad A. Al-Rousan,et al.  Automatic Recognition of Arabic Sign Language Finger Spelling , 2001, Int. J. Comput. Their Appl..

[12]  Alaa El. Sagheer,et al.  An Effective Face Detection Algorithm Based on Skin Color Information , 2012, 2012 Eighth International Conference on Signal Image Technology and Internet Based Systems.

[13]  T. O. Halawani,et al.  Automation of the Arabic sign language recognition , 2004, Proceedings. 2004 International Conference on Information and Communication Technologies: From Theory to Applications, 2004..

[14]  P. K. Mishra.  Superior Skin Color Model using Multiple of Gaussian Mixture Model , 2012 .

[15]  Khaled Assaleh,et al.  Recognition of Arabic Sign Language Alphabet Using Polynomial Classifiers , 2005, EURASIP J. Adv. Signal Process..

[16]  Wen Gao,et al.  A Chinese sign language recognition system based on SOFM/SRN/HMM , 2004, Pattern Recognit..

[17]  Sung Kwan Kang,et al.  Color Based Hand and Finger Detection Technology for User Interaction , 2008, 2008 International Conference on Convergence and Hybrid Information Technology.

[18]  Tamer Shanableh,et al.  Spatio-Temporal Feature-Extraction Techniques for Isolated Gesture Recognition in Arabic Sign Language , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[19]  Mohamed A. Deriche,et al.  Arabic Sign Language Recognition an Image-Based Approach , 2007, 21st International Conference on Advanced Information Networking and Applications Workshops (AINAW'07).