Evaluation of the prototype mobile phone app Pugh: a 3D cartoon character designed to help deaf children to speech read

Pugh, a 3D cartoon character, is a prototype smartphone application developed at the University of Salford. Its purpose is to provide speech-reading exercises for hard of hearing and deaf children. This paper discusses the design of the application, the test process and acknowledges that the technological limitations of the platform and the character’s non-human characteristics provide some interesting challenges. A preliminary test was conducted to evaluate speech perception and lipreading from Pugh. The findings proved that Pugh is not an accurate speaker. Further development of the lip movement and facial expressions is required in order to achieve accuracy.

[1]  M. Coleman,et al.  Speechreading and its association with reading among deaf, hearing and dyslexic individuals , 2006, Clinical linguistics & phonetics.

[2]  R. Campbell,et al.  Hearing by eye 2 : advances in the psychology of speechreading and auditory-visual speech , 1997 .

[3]  THE MANCHESTER SPEECHREADING (LIPREADING) TEST , 1980 .

[4]  Karin Humphreys,et al.  The psychology of language : from data to theory , 2001 .

[5]  Dorothy M. Chun,et al.  Effects of Multimedia Annotations on Vocabulary Acquisition , 1996 .

[6]  Allison Druin,et al.  Robots for Kids: Exploring New Technologies for Learning , 2000 .

[7]  Nicoletta Adamo-Villani,et al.  SMILE: an immersive learning game for deaf and hearing children , 2007, SIGGRAPH '07.

[8]  Dominic W. Massaro,et al.  A Computer Animated Tutor for Language Learning: Research and Applications , 2005 .

[9]  Ruth Campbell,et al.  Speechreading for information gathering: a survey of scientific sources , 2010 .

[10]  L. Bernstein,et al.  Speech perception without hearing , 2000, Perception & psychophysics.

[11]  L. Barker,et al.  Computer-Assisted Vocabulary Acquisition: The CSLU Vocabulary Tutor in Oral-Deaf Education. , 2003, Journal of deaf studies and deaf education.

[12]  Nicoletta Adamo-Villani A Virtual Learning Environment for Deaf Children: Design and Evaluation , 2007 .

[13]  R. Mayberry,et al.  Gesture Reflects Language Development , 2000 .

[14]  Susan Goldin-Meadow,et al.  From children's hands to adults' ears: gesture's role in the learning process. , 2003, Developmental psychology.

[15]  Dominic W. Massaro,et al.  A computer-animated tutor for spoken and written language learning , 2003, ICMI '03.