Avatar-based Sign Language Training Interface for Primary School Education

The aim of this study is to present usable educational tools of students with special needs. Hearing impaired students cannot use text and speech based technological, educational material that is becoming a crucial tool for modern education. Like videos of signers, a signing avatar for TİD (Turkish Sign Language) would allow communicating information in the form of visual gestures, thus becoming usable when the use of text is unfeasible. However, compared to videos, animated avatars offer several advantages like easy reproducible of gesture sequences, control of point of view, adjusting the speed of the sign and being smaller in storage and bandwidth then videos. For this study, a signing avatar was developed to represent a portion of the social sciences course for Turkish primary education curriculum. To analyze the success and effectiveness of the interface, the performance of the text, and avatar-based interaction in a humancomputer interaction scheme using elementary school students have been compared. The results demonstrate that avatar based tutoring was more effective in assessing the child’s knowledge of certain sign language words. Since the aim was to make the signing avatar as comprehensible as possible, the results demonstrate that goal have been.

[1]  Sylvie Gibet,et al.  High level specification and control of communication gestures: the GESSYCA system , 1999, Proceedings Computer Animation 1999.

[2]  Norman I. Badler,et al.  Simulating humans: computer graphics animation and control , 1993 .

[3]  Alexis Héloir,et al.  Sign Language Avatars: Animation and Comprehensibility , 2011, IVA.

[4]  Petros Maragos,et al.  DICTA-SIGN: Sign Language Recognition, Generation and Modelling with application in Deaf Communication , 2010 .

[5]  Hatice Kose-Bagci,et al.  Robostar: An interaction game with humanoid robots for learning sign language , 2014, 2014 IEEE International Conference on Robotics and Biomimetics (ROBIO 2014).

[6]  Nicoletta Adamo-Villani.  A Virtual Learning Environment for Deaf Children: Design and Evaluation , 2007 .

[7]  Mark Wells,et al.  Tessa, a system to aid communication with deaf people , 2002, ASSETS.

[8]  Tomoko Sakiyama,et al.  Pattern Recognition and Synthesis for a Sign Language Translation System , 1996, J. Vis. Lang. Comput..

[9]  S. Powell,et al.  Helping Children with Autism to Learn , 2000 .

[10]  Margriet Verlinden,et al.  A Signing Avatar on the WWW , 2001, Gesture Workshop.

[11]  Hatice Kose-Bagci,et al.  The Effect of Embodiment in Sign Language Tutoring with Assistive Humanoid Robots , 2015, International Journal of Social Robotics.

[12]  Mark Wells,et al.  Virtual signing: capture, animation, storage and transmission-an overview of the ViSiCAST project , 2000 .

[13]  Angus B. Grieve-Smith,et al.  SignSynth: A Sign Language Synthesis Application Using Web3D and Perl , 2001, Gesture Workshop.

[14]  Chrystopher L. Nehaniv,et al.  Title of paper : KASPAR – A Minimally Expressive Humanoid Robot for Human-Robot Interaction Research , 2009 .