Representation of american sign language for machine translation

This dissertation describes an approach to designing a machine translation system that generates a representation of American Sign Language (ASL) from English. ASL uses space and non-manual signals (NMSs) to encode grammatical features such as agreement, negation, wh-questions, etc. Previous computational systems for ASL are typically hindered by static representations of ASL signs, which makes it computationally impractical to represent the large number of possible surface forms for each sign, and by the use of notation systems that cannot represent such variation. The approach developed here addresses these limitations. The representation of ASL is based on the Move-Hold (MH) model (Liddell and Johnson 1989), a sign notation system that allows for both precision of sign description and predictable variation of surface forms based on grammatical detail. Empty features are used in MH notations of lexical forms, which are instantiated with spatial data during generation. The generation system is implemented as an LFG correspondence architecture (Kaplan and Bresnan 1982, Kaplan et al 1989). Correspondence functions are defined that convert an English f-structure into an ASL f-structure; build an ASL c-structure from the f-structure; and build the phonetic representation level (p-structure, where spatial and non-manual variations are revealed) from the c-structure. The concepts presented in this dissertation have been implemented in a software application, ASL Workbench. Possible future applications of this work include developing animated output, tagged corpora for linguistic analysis, and shared lexicons for gloss standardization, among others.

[1]  Ronald M. Kaplan,et al.  An algorithm for functional uncertainty , 1988, COLING.

[2]  Pamela W. Jordan,et al.  A survey of current paradigms in machine translation , 1999, Adv. Comput..

[3]  Howard Poizner,et al.  Computer graphic modeling of american sign language , 1983, SIGGRAPH.

[4]  Ronald M. Kaplan,et al.  The Formal Architecture of Lexical-Functional Grammar , 1989, J. Inf. Sci. Eng..

[5]  Carol Neidle,et al.  Clausal Structure and a Tier for Grammatical Marking in American Sign Language , 1992, Nordic Journal of Linguistics.

[6]  Bonnie J. Dorr,et al.  Machine Translation Divergences: A Formal Description and Proposed Solution , 1994, CL.

[7]  Scott K. Liddell American Sign Language Syntax , 1981 .

[8]  Per-Kristian Halvorsen Situation Semantics and Semantic Interpretation in Constraint-Based Grammars , 1988, FGCS.

[9]  Geoffrey E. Hinton,et al.  Building adaptive interfaces with neural networks: The glove-talk pilot study , 1990, INTERACT.

[10]  Ronnie B. Wilbur,et al.  American sign language: Linguistic and applied dimensions , 1987 .

[11]  Ronald M. Kaplan,et al.  Projections and Semantic Description in Lexical-Functional Grammar , 1988, FGCS.

[12]  Harold L. Somers,et al.  An introduction to machine translation , 1992 .

[13]  Diane C. Lillo-Martin,et al.  WH-MOVEMENT AND THE POSITION OF SPEC-CP : EVIDENCE FROM AMERICAN SIGN LANGUAGE , 1997 .

[14]  Klaus Netter,et al.  Translation by Structural Correspondences , 1989, EACL.

[15]  Shun Ha Sylvia Wong An Investigation into the Use of Argument Structure and Lexical Mapping Theory for Machine Translation , 1998, PACLIC.

[16]  Harlan Lane,et al.  Everyone Here Spoke Sign Language: Hereditary Deafness on Martha's Vineyard , 1986 .

[17]  Howard Poizner,et al.  A computer program to synthesize American Sign Language , 1982 .

[18]  H. Lane The Mask of Benevolence: Disabling the Deaf Community , 1993 .

[19]  Hirosato Nomura,et al.  Lexical-Functional Transfer: A Transfer Framework in a Machine Translation System Based on LFG , 1986, COLING.

[20]  Oliver Sacks,et al.  Seeing Voices: A Journey Into the World of the Deaf. , 1989 .

[21]  E. Klima The signs of language , 1979 .

[22]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[23]  E. Lynn Jacobowitz,et al.  Signs of Tense in ASL Verbs , 1988 .

[24]  Adam Rogers,et al.  Chocolate , 1995, SIGGRAPH Computer Animation Festival.

[25]  Scott K. Liddell,et al.  American Sign Language: The Phonological Base , 2013 .

[26]  H. Lane When the Mind Hears: A History of the Deaf , 1984 .