Flat vs. Expressive Storytelling: Young Children’s Learning and Retention of a Social Robot’s Narrative

Prior research with preschool children has established that dialogic or active book reading is an effective method for expanding young children’s vocabulary. In this exploratory study, we asked whether similar benefits are observed when a robot engages in dialogic reading with preschoolers. Given the established effectiveness of active reading, we also asked whether this effectiveness was critically dependent on the expressive characteristics of the robot. For approximately half the children, the robot’s active reading was expressive; the robot’s voice included a wide range of intonation and emotion (Expressive). For the remaining children, the robot read and conversed with a flat voice, which sounded similar to a classic text-to-speech engine and had little dynamic range (Flat). The robot’s movements were kept constant across conditions. We performed a verification study using Amazon Mechanical Turk (AMT) to confirm that the Expressive robot was viewed as significantly more expressive, more emotional, and less passive than the Flat robot. We invited 45 preschoolers with an average age of 5 years who were either English Language Learners (ELL), bilingual, or native English speakers to engage in the reading task with the robot. The robot narrated a story from a picture book, using active reading techniques and including a set of target vocabulary words in the narration. Children were post-tested on the vocabulary words and were also asked to retell the story to a puppet. A subset of 34 children performed a second story retelling 4–6 weeks later. Children reported liking and learning from the robot a similar amount in the Expressive and Flat conditions. However, as compared to children in the Flat condition, children in the Expressive condition were more concentrated and engaged as indexed by their facial expressions; they emulated the robot’s story more in their story retells; and they told longer stories during their delayed retelling. Furthermore, children who responded to the robot’s active reading questions were more likely to correctly identify the target vocabulary words in the Expressive condition than in the Flat condition. Taken together, these results suggest that children may benefit more from the expressive robot than from the flat robot.

[1]  P. Ekman,et al.  EMFACS-7: Emotional Facial Action Coding System , 1983 .

[2]  Paul L. Harris,et al.  Insight into the Time Course of Emotion among Western and Chinese Children. , 1985 .

[3]  G. Whitehurst,et al.  Accelerating Language Development through Picture Book Reading. , 1988 .

[4]  G. Whitehurst,et al.  Accelerating language development through picture book reading: A systematic extension to Mexican day care. , 1992 .

[5]  Natalie L. Hedberg,et al.  A Comparison of Early Literacy Skills in Children With Specific Language Impairment and Their Typically Developing Peers , 1999 .

[6]  P. Bloom How children learn the meanings of words , 2000 .

[7]  Monique Sénéchal,et al.  A book reading intervention with preschool children who have limited vocabularies: the benefits of regular reading and dialogic reading , 2000 .

[8]  D. Singer,et al.  Handbook of Children and the Media , 2000 .

[9]  David R. Watson,et al.  The impact of impaired vocal quality on children's ability to process spoken language , 2001, Logopedics, phoniatrics, vocology.

[10]  C. Strong,et al.  Literate Language Features in Spoken Narratives of Children With Typical Language and Children With Language Impairments. , 2001, Language, speech, and hearing services in schools.

[11]  Cynthia Breazeal,et al.  Designing sociable robots , 2002 .

[12]  W. Wong,et al.  On ψ-Learning , 2003 .

[13]  J. Huttenlocher,et al.  Syntactic priming in young children , 2004 .

[14]  Kathryne Mcgrath Speaker,et al.  Storytelling: Enhancing Language Acquisition in Young Children. , 2004 .

[15]  Takayuki Kanda,et al.  Interactive Robots as Social Partners and Peer Tutors for Children: A Field Trial , 2004, Hum. Comput. Interact..

[16]  R. Isbell,et al.  The Effects of Storytelling and Story Reading on the Oral Language Complexity and Story Comprehension of Young Children , 2004 .

[17]  P. Ekman,et al.  What the face reveals : basic and applied studies of spontaneous expression using the facial action coding system (FACS) , 2005 .

[18]  Barbara Dodd,et al.  Is there an effect of dysphonic teachers' voices on children's processing of spoken language? , 2005, Journal of voice : official journal of the Voice Foundation.

[19]  Stuart MacFarlane,et al.  Using the fun toolkit and other survey methods to gather opinions in child computer interaction , 2006, IDC '06.

[20]  Thomas R. Cain,et al.  Culture, ethnicity, and children's facial expressions: a study of European American, Mainland Chinese, Chinese American, and adopted Chinese girls. , 2006, Emotion.

[21]  J. J. Diehl,et al.  Story Recall and Narrative Coherence of High-Functioning Children with Autism Spectrum Disorders , 2006, Journal of abnormal child psychology.

[22]  P. Kuhl Is speech learning 'gated' by the social brain? , 2007, Developmental science.

[23]  Patton O. Tabors,et al.  Dual language and literacy development of Spanish-speaking preschool children. , 2007, Journal of applied developmental psychology.

[24]  I. René J. A. te Boekhorst,et al.  Human approach distances to a mechanical-looking robot with different robot voice styles , 2008, RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication.

[25]  Cynthia Breazeal,et al.  Social Robots that Interact with People , 2008, Springer Handbook of Robotics.

[26]  Javier R. Movellan,et al.  Sociable robot improves toddler vocabulary skills , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[27]  Charles Fernyhough,et al.  Young children's trust in their mother's claims: longitudinal links with attachment security in infancy. , 2009, Child development.

[28]  Terrence J Sejnowski,et al.  Foundations for a New Science of Learning , 2009, Science.

[29]  Po-Yao Chao,et al.  Exploring the Possibility of Using Humanoid Robots as Instructional Tools for Teaching a Second Language in Primary School , 2010, J. Educ. Technol. Soc..

[30]  John Heilmann,et al.  Properties of the narrative scoring scheme using narrative retells in young school-age children. , 2010, American journal of speech-language pathology.

[31]  Pierre Dillenbourg,et al.  Technology for Classroom Orchestration , 2010 .

[32]  Dare A. Baldwin,et al.  Social gating and pedagogy: Mechanisms for learning and implications for robotics , 2010, Neural Networks.

[33]  P. Kuhl Social Mechanisms in Early Language Acquisition: Understanding Integrated Brain Systems Supporting Language , 2011 .

[34]  D Feil-Seifer,et al.  Socially Assistive Robotics , 2011, IEEE Robotics & Automation Magazine.

[35]  Georgina Wardle Children's Social Relationships , 2011 .

[36]  Bruce A. MacDonald,et al.  The Effects of Synthesized Voice Accents on User Perceptions of Robots , 2011, Int. J. Soc. Robotics.

[37]  F. Tanaka,et al.  Children teach a care-receiving robot to promote their learning , 2012, J. Hum. Robot Interact..

[38]  Cynthia Breazeal,et al.  Detecting the Trustworthiness of Novel Partners in Economic Exchange , 2012, Psychological science.

[39]  Friederike Eyssel,et al.  ‘If you sound like me, you must be more human’: On the interplay of robot and user features on human-robot acceptance and anthropomorphism , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[40]  Mary-Anne Williams,et al.  Social Robotics , 2012, Lecture Notes in Computer Science.

[41]  Bilge Mutlu,et al.  Pay attention!: designing adaptive agents that monitor and improve user engagement , 2012, CHI.

[42]  Henry Lieberman,et al.  Textual tinkerability: encouraging storytelling behaviors to foster emergent literacy , 2012, CHI Extended Abstracts.

[43]  Daniel McDuff,et al.  Affectiva-MIT Facial Expression Dataset (AM-FED): Naturalistic and Spontaneous Facial Expressions Collected "In-the-Wild" , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[44]  Haizhou Li,et al.  Making Social Robots More Attractive: The Effects of Voice Pitch, Humor and Empathy , 2013, Int. J. Soc. Robotics.

[45]  Heather E. Gary,et al.  Children's Social Relationships With Current and Near‐Future Robots , 2013 .

[46]  Jacqueline Kory,et al.  Storytelling with robots : effects of robot language level on children's language learning , 2014 .

[47]  Ayanna M. Howard,et al.  Using a shared tablet workspace for interactive demonstrations during human-robot learning scenarios , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[48]  Cynthia Breazeal,et al.  Storytelling with robots: Learning companions for preschool children's language development , 2014, The 23rd IEEE International Symposium on Robot and Human Interactive Communication.

[49]  Ana Paiva,et al.  Empathic Robots for Long-term Interaction , 2014, Int. J. Soc. Robotics.

[50]  Cynthia Breazeal,et al.  Can Children Catch Curiosity from a Social Robot? , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[51]  Vanessa Lobue,et al.  The Child Affective Facial Expression (CAFE) set: validity and reliability from untrained adults , 2014, Front. Psychol..

[52]  Daniel McDuff,et al.  Facial Action Unit Detection Using Active Learning and an Efficient Non-linear Kernel Approximation , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[53]  Daniel McDuff,et al.  Crowdsourcing facial responses to online videos: Extended abstract , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[54]  Anara Sandygulova,et al.  Children's Perception of Synthesized Voice: Robot's Gender, Age and Accent , 2015, ICSR.

[55]  Ayanna M. Howard,et al.  Retrieving experience: Interactive instance-based learning methods for building robot companions , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[56]  Cynthia Breazeal,et al.  The Interplay of Robot Language Level with Children's Language Learning during Storytelling , 2015, HRI.

[57]  David S. Nuñez,et al.  GlobalLit : a platform for collecting, analyzing, and reacting to children's usage data on tablet computers , 2015 .

[58]  Magnus Haake,et al.  Does the speaker's voice quality influence children's performance on a language comprehension test? , 2015, International journal of speech-language pathology.

[59]  Heather Pon-Barry,et al.  Effects of voice-adaptation and social dialogue on perceptions of a robotic learning companion , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[60]  Tony Belpaeme,et al.  From characterising three years of HRI to methodology and reporting recommendations , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[61]  Cynthia Breazeal,et al.  Affect-Aware Student Models for Robot Tutors , 2016, AAMAS.

[62]  Daniel McDuff,et al.  AFFDEX SDK: A Cross-Platform Real-Time Multi-Face Expression Recognition Toolkit , 2016, CHI Extended Abstracts.

[63]  Cynthia Breazeal,et al.  Fostering parent–child dialog through automated discussion suggestions , 2016, User Modeling and User-Adapted Interaction.

[64]  Tony Belpaeme,et al.  Social robot tutoring for child second language learning , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[65]  Cynthia Breazeal,et al.  Affective Personalization of a Social Robot Tutor for Children's Second Language Skills , 2016, AAAI.

[66]  Cynthia Breazeal,et al.  Young Children Treat Robots as Informants , 2016, Top. Cogn. Sci..

[67]  Cynthia Breazeal,et al.  Tega: A social robot , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[68]  Tony Belpaeme,et al.  Nonverbal Immediacy as a Characterisation of Social Behaviour for Human–Robot Interaction , 2016, International Journal of Social Robotics.