New tools for interactive speech and language training: Using animated conversational agents in the classrooms of profoundly deaf children

This article describes our experiences with an animated conversational agent being used in daily classroom activities with profoundly deaf children at the Tucker Maxon Oral School in Portland Oregon. We first articulate some reasons why animated conversational agents can revolutionize learning and language training by providing a more effective mode of human computer interaction. We then describe the capabilities of our animated agent, Baldi, and the software environment used to design and run interactive media systems. We then describe applications designed by teachers and students that illustrate ways in which students in three different classrooms converse and interact with Baldi. We conclude with a brief look at the next generation of animated conversational agents.

[1]  Alex Waibel,et al.  Intelligent animated agents for interactive language training , 1998, SIGC.

[2]  Ronald A. Cole,et al.  Creating a mexican Spanish version of the CSLU toolkit , 1998, ICSLP.

[3]  Yonghong Yan,et al.  Universal speech tools: the CSLU toolkit , 1998, ICSLP.

[4]  Paul Taylor,et al.  Festival Speech Synthesis System , 1998 .

[5]  M. Stone,et al.  Three-dimensional tongue surface shapes of English consonants and vowels. , 1996, The Journal of the Acoustical Society of America.

[6]  Yonghong Yan,et al.  Accessible technology for interactive systems: a new approach to spoken language research , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[7]  G. Plant Perceiving Talking Faces: From Speech Perception to a Behavioral Principle , 1999 .

[8]  Clifford Nass,et al.  The media equation - how people treat computers, television, and new media like real people and places , 1996 .

[9]  D. Massaro,et al.  Perceiving Talking Faces , 1995 .

[10]  Michael Johnston,et al.  PROFER: predictive, robust finite-state parsing for spoken language , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).