Robust Sign Language Recognition System Using ToF Depth Cameras

Sign language recognition is a difficult task, yet required for many applications in real-time speed. Using RGB cameras for recognition of sign languages is not very successful in practical situations and accurate 3D imaging requires expensive and complex instruments. With introduction of Time-of-Flight (ToF) depth cameras in recent years, it has become easier to scan the environment for accurate, yet fast depth images of the objects without the need of any extra calibrating object. In this paper, a robust system for sign language recognition using ToF depth cameras is presented for converting the recorded signs to a standard and portable XML sign language named SiGML for easy transferring and converting to real-time 3D virtual characters animations. Feature extraction using moments and classification using nearest neighbor classifier are used to track hand gestures and significant result of 100% is achieved for the proposed approach.

[1]  Robert W. Lindeman,et al.  A multi-class pattern recognition system for practical finger spelling translation , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[2]  Michael Filhol,et al.  Sign Language Corpora for Analysis, Processing and Evaluation , 2010, LREC.

[3]  Hideo Saito,et al.  Virtual 3-D interface system via hand motion recognition from two cameras , 2002, IEEE Trans. Syst. Man Cybern. Part A.

[4]  Hermann Ney,et al.  Smoothed Disparity Maps for Continuous American Sign Language Recognition , 2009, IbPRIA.

[5]  Hermann Ney,et al.  Benchmark Databases for Video-Based Automatic Sign Language Recognition , 2008, LREC.

[6]  Hermann Ney,et al.  Using Different Aspects of the Signings for Appearance-based Sign Language Recognition , 2008 .

[7]  Syed Atif Mehdi,et al.  Sign language recognition using sensor gloves , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[8]  Hermann Ney,et al.  The SignSpeak Project - Bridging the Gap Between Signers and Speakers , 2010, LREC.

[9]  Javier Macías Guarasa,et al.  Speech to sign language translation system for Spanish , 2008, Speech Commun..

[10]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.

[11]  Michael G. Strintzis,et al.  A gesture recognition system using 3D data , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.