Study of sign segmentation in the text of Chinese sign language

The natural language processing (NLP) of sign language aims to make human sign language “understandable” to computers. In achieving this goal, the text of sign language should first be segmented into sign sequences for computers to recognize. This segmentation process constitutes the basis for the information processing of sign language. With an aim to solve the problems in expressing Chinese sign language (CSL), this paper analyzes the lexical features of CSL and discusses various sign segmentation algorithms used in obtaining computer-read files. Sign segmentation involves two main approaches: The first is rule based, whereas the second is statistics based. Backward maximum matching (BMM) is an important rule-based method widely used in Chinese NLP fields. The recently proposed conditional random fields (CRFs) have also demonstrated excellent performance as a statistical method in international tests. In this study, both the BMM and CRFs methods are employed on the same dataset to explore the practical issues in the sign segmentation of CSL. The results of the CRFs method are then presented and discussed. Our corpus contains only hundreds of sentences; therefore, cross-validation based on CRFs is also performed to avoid the unreliable function that may arise from using an exceedingly small corpus scale within limited processing time. Specifically, three-group twofold cross-validation is applied to analyze the design of the annotation specification and the selection of a feature template. The results validate the effectiveness of our proposed segmentation strategy and confirm that CRFs outperform the BMM method. The proposed approach yields an F-score of 77.4% in sign segmentation in the CSL corpus. The CRFs perform effectively in sign segmentation because they can capture the arbitrary, overlapping features of the input in a Markov model. However, to obtain more satisfactory results, we must rely on the technological development of the sign language corpus.

[1]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[2]  Yoshua Bengio,et al.  Inference for the Generalization Error , 1999, Machine Learning.

[3]  Hermann Ney,et al.  Morpho-Syntax Based Statistical Methods for Automatic Sign Language Translation , 2006, EAMT.

[4]  Scott K. Liddell.  Grammar, Gesture, and Meaning in American Sign Language , 2003 .

[5]  Yuji Matsumoto,et al.  Applying Conditional Random Fields to Japanese Morphological Analysis , 2004, EMNLP.

[6]  Ian Marshall,et al.  Grammar Development for Sign Language Avatar-Based Synthesis , 2005 .

[7]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[8]  T. Johnston.  W(h)ither the Deaf Community? Population, Genetics, and the Future of Australian Sign Language , 2004, American annals of the deaf.

[9]  Hermann Hienz,et al.  Relevant features for video-based continuous sign language recognition , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[10]  Andrew McCallum,et al.  Maximum Entropy Markov Models for Information Extraction and Segmentation , 2000, ICML.

[11]  Wang Yu,et al.  Blocked 3×2 Cross-Validated t-Test for Comparing Supervised Classification Learning Algorithms , 2014, Neural Comput..

[12]  Tianshun Yao,et al.  Applying Conditional Random Fields to Chinese Shallow Parsing , 2005, CICLing.

[13]  Trevor Johnston,et al.  finish variation and grammaticalization in a signed language: How far down this well-trodden pathway is Auslan (Australian Sign Language)? , 2015, Language Variation and Change.

[14]  Sylvain Arlot.  TECHNICAL APPENDIX TO "V -FOLD CROSS-VALIDATION IMPROVED: V -FOLD PENALIZATION , 2008, 0802.0566.

[15]  Xiaotie Deng,et al.  Unsupervised Segmentation of Chinese Corpus Using Accessor Variety , 2004, IJCNLP.

[16]  Matt Huenerfauth.  American Sign Language Generation: Multimodal NLG with Multiple Linguistic Channels , 2005, ACL.

[17]  Richard Bowden,et al.  Sign Language Recognition , 2011, Visual Analysis of Humans.

[18]  Yu Shi-wen.  The Basic Processing of Contemporary Chinese Corpus at Peking University SPECIFICATION , 2002 .

[19]  Hermann Ney,et al.  Morpho-syntax Based Statistical Methods for Sign Language Translation vorgelegt von : Cand , 2006 .

[20]  W. Bruce Croft,et al.  Table extraction using conditional random fields , 2003, DG.O.

[21]  Hai Zhao.  Incorporating Global Information into Supervised Learning for Chinese Word Segmentation , 2007 .

[22]  Sherman Wilcox,et al.  Learning To See: Teaching American Sign Language as a Second Language , 1997 .

[23]  Norman I. Badler,et al.  A machine translation system from English to American Sign Language , 2000, AMTA.

[24]  Jordan Fenlon,et al.  Building BSL SignBank: The Lemma Dilemma Revisited , 2015 .

[25]  Herbert Schildt.  Natural-language processing in C , 1987 .

[26]  H. Ney,et al.  Towards Automatic Sign Language Annotation for the ELAN Tool , 2008 .