Towards adapting fantasy, curiosity and challenge in multimodal dialogue systems for preschoolers

We investigate how fantasy, curiosity and challenge contribute to the user experience in multimodal dialogue computer games for preschool children. For this purpose, an on-line multimodal platform has been designed, implemented and used as a starting point to develop web-based speech-enabled applications for children. Five task oriented games suitable for preschoolers have been implemented with varying levels of fantasy and curiosity elements, as well as, variable difficulty levels. Nine preschool children, ages 4-6, were asked to play these games in three sessions; in each session only one of the fantasy, curiosity or challenge factor was evaluated. Both objective and subjective criteria were used to evaluate the factors and applications. Results show that fantasy and curiosity are correlated with children's entertainment, while the level of difficulty seems to depend on each child's individual preferences and capabilities. In addition, high speech usage and high curiosity levels in the application correlate well with task completion, showing that preschoolers become more engaged when multimodal interfaces are speech enabled and contain curiosity elements.

[1]  Justine Cassell,et al.  Making Space for Voice: Technologies to Support Children’s Fantasy and Storytelling , 2001, Personal and Ubiquitous Computing.

[2]  N. Lazzaro.  Why we Play Games: Four Keys to More Emotion without Story , 2004 .

[3]  H. Ruff,et al.  Development of sustained, focused attention in young children during free play. , 1990 .

[4]  Jack Mostow,et al.  Demonstration of a reading coach that listens , 1995, UIST '95.

[5]  Amy Bruckman,et al.  Human-computer interaction for kids , 2002 .

[6]  A. U.S..  MULTIMODAL SYSTEMS FOR CHILDREN : BUILDING A PROTOTYPE , 1999 .

[7]  Shrikanth S. Narayanan,et al.  Robust recognition of children's speech , 2003, IEEE Trans. Speech Audio Process..

[8]  Joakim Gustafson,et al.  The NICE Fairy-tale Game System , 2004, SIGDIAL Workshop.

[9]  Shrikanth S. Narayanan,et al.  Creating conversational interfaces for children , 2002, IEEE Trans. Speech Audio Process..

[10]  Georgios N. Yannakakis,et al.  Feature selection for capturing the experience of fun , 2007 .

[11]  Georgios N. Yannakakis,et al.  Comparative fun analysis in the innovative playware game platform , 2006 .

[12]  M. Csíkszentmihályi.  Flow. The Psychology of Optimal Experience. New York (HarperPerennial) 1990. , 1990 .

[13]  Bryan L. Pellom,et al.  Children's speech recognition with application to interactive books and tutors , 2003, 2003 IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No.03EX721).

[14]  J. Jacko,et al.  The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications , 2002 .

[15]  Georgios N. Yannakakis,et al.  Capturing Entertainment Through Heart Rate Dynamics in the Playware Playground , 2006, ICEC.

[16]  Guillaume Chanel,et al.  Boredom, engagement and anxiety as indicators for adaptation to difficulty in games , 2008, MindTrek '08.

[17]  Thomas W. Malone,et al.  What Makes Things Fun to Learn? A Study of Intrinsically Motivating Computer Games. , 1981 .

[18]  Nicole Beringer,et al.  PROMISE - A Procedure for Multimodal Interactive System Evaluation , 2002 .

[19]  Shrikanth S. Narayanan,et al.  Acoustics of children's speech: developmental changes of temporal and spectral parameters. , 1999, The Journal of the Acoustical Society of America.

[20]  Georgios N. Yannakakis,et al.  Evolving opponents for interesting interactive computer games , 2004 .

[21]  F. Frome,et al.  Talking back to big bird: Preschool users and a simple speech recognition system , 1993 .

[22]  Sharon L. Oviatt,et al.  Multimodal integration patterns in children , 2002, INTERSPEECH.

[23]  Zhigang Deng,et al.  Analysis of emotion recognition using facial expressions, speech and multimodal information , 2004, ICMI '04.

[24]  J. Piaget.  Science of education and the psychology of the child , 1970 .

[25]  Martin J. Russell,et al.  Applications of automatic speech recognition to speech and language development in young children , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[26]  Alexandros Potamianos,et al.  A Study in Efficiency and Modality Usage in Multimodal Form Filling Systems , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[27]  Georgios N. Yannakakis,et al.  Entertainment Modeling in Physical Play Through Physiology Beyond Heart-Rate , 2007, ACII.

[28]  Shrikanth S. Narayanan,et al.  Detecting Politeness and frustration state of a child in a conversational computer game , 2005, INTERSPEECH.