Smoothed Disparity Maps for Continuous American Sign Language Recognition

For the recognition of continuous sign language we analyse whether we can improve the results by explicitly incorporating depth information. Accurate hand tracking for sign language recognition is made difficult by abrupt and fast changes in hand position and configuration, overlapping hands, or a hand signing in front of the face. In our system depth information is extracted using a stereo-vision method that considers the time axis by using pre- and succeeding frames. We demonstrate that depth information helps to disambiguate overlapping hands and thus to improve the tracking of the hands. However, the improved tracking has little influence on the final recognition results.

[1]  Xin Liu,et al.  Real Time Large Vocabulary Continuous Sign Language Recognition Based on OP/Viterbi Algorithm , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[2]  Trevor Darrell,et al.  Hidden Conditional Random Fields for Gesture Recognition , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[3]  Wen Gao,et al.  Large-Vocabulary Continuous Sign Language Recognition Based on Transition-Movement Models , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[4]  Takeo Kanade,et al.  Stereo by Intra- and Inter-Scanline Search Using Dynamic Programming , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Xia Liu,et al.  Sign recognition using depth image streams , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[6]  E. A. Hendriks,et al.  3 D Visual Detection of Correct NGT Sign Production , 2007 .

[7]  Andrew Blake,et al.  Probabilistic Fusion of Stereo with Color and Contrast for Bilayer Segmentation , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Surendra Ranganath,et al.  Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Hermann Ney,et al.  Speech recognition techniques for a sign language recognition system , 2007, INTERSPEECH.

[10]  Andrew W. Fitzgibbon,et al.  Learning priors for calibrating families of stereo cameras , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[11]  Hermann Ney,et al.  Tracking using dynamic programming for appearance-based sign language recognition , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[12]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[13]  Ulrike Rosa Wrobel.  Referenz in Gebärdensprachen: Raum und Person , 2001 .

[14]  Andrew Blake,et al.  Probabilistic Fusion of Stereo with Color and Contrast for Bi-Layer Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Dimitris N. Metaxas,et al.  A Framework for Recognizing the Simultaneous Aspects of American Sign Language , 2001, Comput. Vis. Image Underst..

[16]  Ruiduo Yang,et al.  Enhanced Level Building Algorithm for the Movement Epenthesis Problem in Sign Language Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.