Chinese Sign Language Recognition Based on SHS Descriptor and Encoder-Decoder LSTM Model

This paper presents a novel approach to recognize isolated Chinese sign language. In order to better distinguish different hand shapes, a new Specific Hand Shape (SHS) descriptor is proposed. Based on the SHS descriptor, an encoder-decoder LSTM model is applied to achieve better sign recognition results. A specific hand shape database and an 80 words isolated Chinese sign language database are constructed using Kinect 2.0 to evaluate the proposed methods. Experimental results show the proposed SHS descriptor is more discriminative than the traditional HOG descriptor and the recognition model is more efficient than the HMM based approach.

[1]  Raúl Rojas,et al.  Sign Language Recognition Using Kinect , 2012, ICAISC.

[2]  Sander Dieleman,et al.  Beyond Temporal Pooling: Recurrence and Temporal Convolutions for Gesture Recognition in Video , 2015, International Journal of Computer Vision.

[3]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[4]  Yangsheng Xu,et al.  Online, interactive learning of gestures for human/robot interfaces , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[5]  Houqiang Li,et al.  Sign language recognition with long short-term memory , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[6]  Zhongfu Ye,et al.  An Isolated Sign Language Recognition System Using RGB-D Sensor with Sparse Coding , 2014, 2014 IEEE 17th International Conference on Computational Science and Engineering.

[7]  Trevor Darrell,et al.  Sequence to Sequence -- Video to Text , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Ao Tang,et al.  A Real-Time Hand Posture Recognition System Using Deep Neural Networks , 2015, ACM Trans. Intell. Syst. Technol..

[9]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[10]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Houqiang Li,et al.  Sign Language Recognition using 3D convolutional neural networks , 2015, 2015 IEEE International Conference on Multimedia and Expo (ICME).

[12]  Thad Starner,et al.  American sign language recognition with the kinect , 2011, ICMI '11.

[13]  Benjamin Schrauwen,et al.  Sign Language Recognition Using Convolutional Neural Networks , 2014, ECCV Workshops.