Why Don't You See What I Mean? Prospects and Limitations of Current Automatic Sign Recognition Research

This article presents an overview of current automatic sign recognition research. A review of recent studies, as well as on our own research, has identified several problem areas that hamper successful sign recognition by a computer. Some of these problems are shared with automatic speech recognition, whereas others seem to be unique to automatic sign recognition. These latter difficulties include context dependency, determination of the basic units of modeling, the ability to distinguish between signs and gestures, movement epenthesis, and repetition within signs. As a possible solution to these problems, bottom-up processing should perhaps be supplemented with top-down processing.

[1]  G. T. Holt.  Automatic recognition of Dutch sign language , 2004 .

[2]  Mark Wells,et al.  Tessa, a system to aid communication with deaf people , 2002, ASSETS.

[3]  Kirsti Grobel,et al.  Isolated sign language recognition using hidden Markov models , 1996, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[4]  Wendy Sandler,et al.  Phonological features and feature classes: The case of movements in sign language , 1996 .

[5]  Melanie Metzger,et al.  Gesture in sign language discourse , 1998 .

[6]  David M. Perlmutter.  SONORITY AND SYLLABLE STRUCTURE IN AMERICAN SIGN LANGUAGE , 1993 .

[7]  Karl-Friedrich Kraiss,et al.  Video-based sign recognition using self-organizing subunits , 2002, Object recognition supported by user interaction for service robots.

[8]  Gwen Littlewort,et al.  Machine learning methods for fully automatic recognition of facial expressions and facial actions , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[9]  Scott K. Liddell,et al.  American Sign Language: The Phonological Base , 2013 .

[10]  Ying-li Tian,et al.  Evaluation of Face Resolution for Expression Analysis , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[11]  David Windridge,et al.  A Linguistic Feature Vector for the Visual Interpretation of Sign Language , 2004, ECCV.

[12]  M. B. Waldron,et al.  Isolated ASL sign recognition system for deaf persons , 1995 .

[13]  Alex Acero,et al.  Spoken Language Processing: A Guide to Theory, Algorithm and System Development , 2001 .

[14]  U. Bellugi,et al.  What the hands reveal about the brain , 1987 .

[15]  Mark Wells,et al.  The Development and Evaluation of a Speech-to-Sign Translation System to Assist Transactions , 2003, Int. J. Hum. Comput. Interact..

[16]  Dimitris N. Metaxas,et al.  Handshapes and movements: Multiple-channel ASL recognition , 2004 .

[17]  Ming Ouhyoung,et al.  A real-time continuous gesture recognition system for sign language , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[18]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Dimitris N. Metaxas,et al.  Toward Scalability in ASL Recognition: Breaking Down Signs into Phonemes , 1999, Gesture Workshop.

[20]  Hermann Hienz,et al.  Video-based continuous sign language recognition using statistical methods , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[21]  Harry van der Hulst,et al.  Issues in sign linguistic: Phonetics, phonology and morpho-syntax☆ , 1996 .

[22]  Dimitris N. Metaxas,et al.  Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[23]  Wen Gao,et al.  CSLDS: Chinese sign language dialog system , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[24]  Mohammed Waleed Kadous,et al.  Machine Recognition of Auslan Signs Using PowerGloves: Towards Large-Lexicon Recognition of Sign Lan , 1996 .

[25]  E. Klima.  The signs of language , 1979 .