Towards Automated Visualization and Analysis of Signed Language Motion: Method and Linguistic Issues

This paper presents a semi-automatic method for the visualization and analysis of motion in signed language from a digital video. The method detects the different parts of the signer’s bare skin on the video, tracks the motion of the different parts of the body and represents the frame-wise motion of the parts of the body with statistical descriptors. Results of an experiment are demonstrated and discussed. The main linguistic issue deals with the relationship between signs and transitional sequences. Evidence is provided for the claim that lexical-phonological and transitional sequences are qualitatively different in terms of their motion characteristics.

[1]  Tommi Jantunen,et al.  Sign Languages: Syllable structure in sign language phonology , 2010 .

[2]  Robert E. Johnson,et al.  American Sign Language compound formation processes, lexicalization, and phonological remnants , 1986 .

[3]  Tommi Jantunen.  Tavu ja lause tutkimuksia kahden sekventiaalisen perusyksikön olemuksesta suomalaisessa viittomakielessä , 2008 .

[4]  Tommi.  Jantunen Fixed and Free : Order of the Verbal Predicate and Its Core Arguments in Declarative Transitive Clauses in Finnish Sign Language , 2009 .

[5]  Sherman Wilcox,et al.  The phonetics of fingerspelling , 1992 .

[6]  O. Crasborn,et al.  Syllables and the word-prosodic system in Sign Language of the Netherlands , 2008 .

[7]  Tommi Jantunen,et al.  TAVU SUOMALAISESSA VIITTOMAKIELESSÄ , 2012 .

[8]  David M. Perlmutter.  SONORITY AND SYLLABLE STRUCTURE IN AMERICAN SIGN LANGUAGE , 1993 .

[9]  Ulrike Zeshan,et al.  Towards a Notion of ‘Word’ in Sign Languages , 2002 .

[10]  M. Nespor,et al.  Prosody in Israeli Sign Language , 1999 .

[11]  Ronnie B. Wilbur,et al.  Phonological and prosodic layering of nonmanuals in American Sign Language. , 2000 .

[12]  P. B. Braem.  Rhythmic Temporal Patterns in the Signing of Deaf Early and Late Learners of Swiss German Sign Language , 1999 .

[13]  Ronnie B. Wilbur,et al.  Aspects of Rhythm in ASL , 2013 .

[14]  Diane Brentari,et al.  A Prosodic Model of Sign Language Phonology , 1999 .

[15]  E. Klima.  The signs of language , 1979 .

[16]  Tommi Jantunen,et al.  1 SYLLABLE STRUCTURE IN SIGN LANGUAGE PHONOLOGY , 2008 .

[17]  Wendy Sandler,et al.  Cilticization and Prosodic Words in a Sign Language , 1999 .

[18]  P. Siple,et al.  Theoretical issues in sign language research , 1990 .