An Isolated Sign Language Recognition System Using RGB-D Sensor with Sparse Coding

An isolated sign language recognition system is presented in this paper. A RGB-D sensor, Microsoft Kinect, is used for obtaining color stream and skeleton points from the depth stream. For a particular sign we extract a representative feature vector composed by hand trajectories and hand shapes. A sparse dictionary learning algorithm, Label Consistent K-SVD (LC-KSVD), is applied to obtain a discriminative dictionary. Based on that, we further develop a new classification approach to get better result. Our system is evaluated on 34 isolated Chinese sign words including one-handed signs and two-handed signs. Experimental results show the proposed system gets high recognition accuracy, of the reported 96.75%, and obtain an average accuracy of 92.36% for signer independent recognition.

[1]  Sergio Escalera,et al.  Featureweighting in dynamic timewarping for gesture recognition in depth data , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[2]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[3]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[4]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[5]  Tanaya Guha,et al.  Learning Sparse Representations for Human Action Recognition , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Robyn A. Owens,et al.  Australian sign language recognition , 2005, Machine Vision and Applications.

[7]  Larry S. Davis,et al.  Learning a discriminative dictionary for sparse coding via label consistent K-SVD , 2011, CVPR 2011.

[8]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[9]  Surendra Ranganath,et al.  Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  M. Kavakli,et al.  A robust gesture recognition algorithm based on Sparse Representation, random projections and Compressed Sensing , 2012, 2012 7th IEEE Conference on Industrial Electronics and Applications (ICIEA).

[11]  Wen Gao,et al.  Sign Language Recognition Based on HMM/ANN/DP , 2000, Int. J. Pattern Recognit. Artif. Intell..

[12]  Raúl Rojas,et al.  Sign Language Recognition Using Kinect , 2012, ICAISC.

[13]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  Thad Starner,et al.  American sign language recognition with the kinect , 2011, ICMI '11.

[15]  Baoxin Li,et al.  Discriminative K-SVD for dictionary learning in face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  B. Watanapa,et al.  Human gesture recognition using Kinect camera , 2012, 2012 Ninth International Conference on Computer Science and Software Engineering (JCSSE).