A novel approach to automatically extracting basic units from Chinese sign language

In sign language recognition, using subwords instead of whole signs as basic units scales well with increasing vocabulary size. However, there are no subwords defined in the signs' lexical forms. How to automatically extract subwords is a challenging issue. In this paper, a novel approach is proposed to automatically extract these subwords from Chinese sign language (CSL). Signs can be broken down into several segments using hidden Markov models in which each state represents one segment. Temporal clustering algorithm is presented to extract subwords from these segments. The 238 subwords are automatically extracted from 5113 signs, and they can be used as the basic units for large vocabulary CSL recognition with good performance.

[1]  Dimitris N. Metaxas,et al.  Toward Scalability in ASL Recognition: Breaking Down Signs into Phonemes , 1999, Gesture Workshop.

[2]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Dimitris N. Metaxas,et al.  A Framework for Recognizing the Simultaneous Aspects of American Sign Language , 2001, Comput. Vis. Image Underst..

[4]  Lawrence R. Rabiner,et al.  A modified K-means clustering algorithm for use in isolated work recognition , 1985, IEEE Trans. Acoust. Speech Signal Process..

[5]  Kirsti Grobel,et al.  Isolated sign language recognition using hidden Markov models , 1996, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[6]  W. Stokoe,et al.  Sign language structure: an outline of the visual communication systems of the American deaf. 1960. , 1961, Journal of deaf studies and deaf education.

[7]  Dimitris N. Metaxas,et al.  ASL recognition based on a coupling between HMMs and 3D motion analysis , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[8]  Ming Ouhyoung,et al.  A real-time continuous gesture recognition system for sign language , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[9]  Karl-Friedrich Kraiss,et al.  Towards an Automatic Sign Language Recognition System Using Subunits , 2001, Gesture Workshop.

[10]  Scott K. Liddell,et al.  American Sign Language: The Phonological Base , 2013 .