Spanish Sign Language synthesis system

This work presents a new approach to the synthesis of Spanish Sign Language (LSE). Its main contributions are the use of a centralized relational database for storing sign descriptions, the proposal of a new input notation and a new avatar design, the skeleton structure of which improves the synthesis process. The relational database facilitates a highly detailed phonologic description of the signs that include parameter synchronization and timing. The centralized database approach has been introduced to allow the representation of each sign to be validated by the LSE National Institution, FCNSE. The input notation, designated HLSML, presents multiple levels of abstraction compared with current input notations. Redesigned input notation is used to simplify the description and the manual definition of LSE messages. Synthetic messages obtained using our approach have been evaluated by deaf users; in this evaluation a maximum recognition rate of 98.5% was obtained for isolated signs and a recognition rate of 95% was achieved for signed sentences.

[1]  Z. Krňoul,et al.  New features in synthesis of sign language addressing non-manual component , 2010 .

[2]  Jaroslaw Francik,et al.  Animating Sign Language in the Real Time , 2002 .

[3]  A. Herrero.  UNA APROXIMACIÓN MORFOLÓGICA A LAS CONSTRUCCIONES CLASIFICATORIAS EN LA LENGUA DE SIGNOS ESPAÑOLA , 2004 .

[4]  Javier Macías Guarasa,et al.  Speech to sign language translation system for Spanish , 2008, Speech Commun..

[5]  M. Inés Torres,et al.  Evaluation of alternatives on speech to sign language translation , 2007, INTERSPEECH.

[6]  Mark Wells,et al.  Virtual signing: capture, animation, storage and transmission-an overview of the ViSiCAST project , 2000 .

[7]  José Colás Pasamontes,et al.  The Synthesis of LSE Classifiers: From Representation to Evaluation , 2011, J. Univers. Comput. Sci..

[8]  Ken Shoemake,et al.  Animating rotation with quaternion curves , 1985, SIGGRAPH.

[9]  Steve C. Maddock,et al.  A Constraint-Based Approach to Visual Speech for a Mexican-Spanish Talking Head , 2008, Int. J. Comput. Games Technol..

[10]  Duan Xinyu,et al.  Three Dimension Human Body Format and Its Virtual Avatar Animation Application , 2008, 2008 Second International Symposium on Intelligent Information Technology Application.

[11]  Michael G. Strintzis,et al.  Gesture synthesis from sign language notation using MPEG-4 humanoid animation parameters and inverse kinematics , 2006 .

[12]  Ian Marshall,et al.  Linguistic modelling and language-processing technologies for Avatar-based sign language presentation , 2008, Universal Access in the Information Society.

[13]  Javier Macías Guarasa,et al.  Generating gestures from speech , 2004, INTERSPEECH.

[14]  Javier Macías Guarasa,et al.  Proposing a speech to gesture translation architecture for Spanish deaf people , 2008, J. Vis. Lang. Comput..

[15]  Ronnie B. Wilbur,et al.  Phonological and prosodic layering of nonmanuals in American Sign Language. , 2000 .

[16]  S. Piotr,et al.  Personalized avatar animation for virtual reality , 2008, 2008 1st International Conference on Information Technology.

[17]  Paolo Prinetto,et al.  A Multilanguage Database for Supporting Sign Language Translation and Synthesis , 2010, LREC 2010.

[18]  Frank E. Ritter,et al.  Model-based evaluation of expert cell phone menu interaction , 2007, TCHI.

[19]  Melanie Metzger,et al.  Gesture in sign language discourse , 1998 .

[20]  Bon-Ki Koo,et al.  An intuitive system for 3D avatar with high-quality , 2010, 2010 Digest of Technical Papers International Conference on Consumer Electronics (ICCE).

[21]  John R. W. Glauert,et al.  Providing signed content on the Internet by synthesized animation , 2007, TCHI.

[22]  Richard Kennaway,et al.  Synthetic Animation of Deaf Signing Gestures , 2001, Gesture Workshop.

[23]  Rubén San-Segundo-Hernández,et al.  A Spanish speech to sign language translation system for assisting deaf-mute people , 2006, INTERSPEECH.

[24]  Franc Solina,et al.  Multimedia dictionary and synthesis of sign language , 2001 .

[25]  Mitchell P. Marcus,et al.  Generating american sign language classifier predicates for english-to-asl machine translation , 2006 .

[26]  Sylvie Gibet,et al.  Heterogeneous Data Sources for Signed Language Analysis and Synthesis: The SignCom Project , 2010, LREC.

[27]  Dorothea Cogill-Koez.  A model of signed language ‘classifier predicates’ as templated visual representation , 2000 .

[28]  Graçaliz Pereira Dimuro,et al.  Supporting Deaf Sign Languages in Written Form on the Web , 2001, WWW Posters.

[29]  Mark Wells,et al.  The Development and Evaluation of a Speech-to-Sign Translation System to Assist Transactions , 2003, Int. J. Hum. Comput. Interact..

[30]  Petros Maragos,et al.  Sign Language Recognition, Generation, and Modelling: A Research Effort with Applications in Deaf Communication , 2009, HCI.

[31]  Matt Huenerfauth.  Spatial Representation of Classifier Predicates for Machine Translation into American Sign Language , 2004 .

[32]  Adam Schembri,et al.  Rethinking ‘Classifiers’ in Signed Languages , 2003 .

[33]  Angus B. Grieve-Smith,et al.  SignSynth: A Sign Language Synthesis Application Using Web3D and Perl , 2001, Gesture Workshop.

[34]  Veronica Lopez Ludeña,et al.  Language Resources for Spanish - Spanish Sign Language (LSE) translation , 2010, LREC 2010.

[35]  Yasunori Yamamoto,et al.  Sign Language Synthesis for the Mobile Environments , 2003, WSCG.

[36]  William Rowan Hamilton,et al.  Lectures on quaternions , 1853 .

[37]  Ángel Herrero Blanco.  Una aproximación morfológica a las construcciones clasificatorias en la lengua de signos española , 2004 .

[38]  Onno Crasborn,et al.  SignPhon: A phonological database for sign languages , 2002 .

[39]  Michael G. Strintzis,et al.  Synthesis of Virtual Reality Animations from SWML using MPEG-4 Body Animation Parameters , 2004 .

[40]  John Glauert,et al.  Requirements For A Signing Avatar , 2010 .