Exploring Children's Verbal and Acoustic Synchrony: Towards Promoting Engagement in Speech-Controlled Robot-Companion Games

Children's interpersonal synchrony has been related to various benefits in social, mental and emotional development. We explore verbal and acoustic synchrony patterns between pairs of children playing a speech-controlled video game. Verbal features include word timing and duration patterns, while acoustic cues contain prosodic information. Synchrony is captured through a random-effects model taking into account multiple sources of variation and repeated measurements for each pair of children. Our findings indicate the presence of synchrony between participants during game play, which increases as they become more engaged in the game. These results are discussed in relation to personalized human-computer interaction and adaptive game environments.

[1]  Kostas Karpouzis,et al.  Fusing Visual and Behavioral Cues for Modeling User Experience in Games , 2013, IEEE Transactions on Cybernetics.

[2]  Shrikanth Narayanan,et al.  ASSESSMENT OF A CHILD ’ S ENGAGEMENT USING SEQUENCE MODEL BASED FEATURES , 2013 .

[3]  Samer Al Moubayed,et al.  Toward Better Understanding of Engagement in Multiparty Spoken Interaction with Children , 2015, ICMI.

[4]  J. Read,et al.  Endurability, Engagement and Expectations: Measuring Children’s Fun , 2002 .

[5]  Heikki Lyytinen,et al.  Children's engagement during digital game-based learning of reading: The effects of time, rewards, and challenge , 2014, Comput. Educ..

[6]  Tal-Chen Rabinowitch,et al.  Synchronous Rhythmic Interaction Enhances Children’s Perceived Similarity and Closeness towards Each Other , 2015, PloS one.

[7]  Victor Minces,et al.  Group rhythmic synchrony and attention in children , 2013, Front. Psychol..

[8]  Candace L. Sidner,et al.  Explorations in engagement for humans and robots , 2005, Artif. Intell..

[9]  Jeffrey T. Hancock,et al.  Language Style Matching as a Predictor of Social Dynamics in Small Groups , 2010, Commun. Res..

[10]  Michael J. Hove,et al.  It's all in the timing: Interpersonal synchrony increases affiliation , 2009 .

[11]  G. Bryant,et al.  Convergence of speech rate in conversation predicts cooperation , 2013 .

[12]  I. Granic,et al.  Dyadic Attunement and Physiological Synchrony During Mother-Child Interactions: An Exploratory Study in Children With and Without Externalizing Behavior Problems , 2015 .

[13]  Yan Peng Lim,et al.  Children, engagement and enjoyment in digital narrative , 2010 .

[14]  James M. Rehg,et al.  Decoding Children's Social Behavior , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Joseph Hilbe,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2009 .

[16]  Björn Schuller,et al.  Opensmile: the munich versatile and fast open-source audio feature extractor , 2010, ACM Multimedia.

[17]  Margaret M Kehoe.  Prosodic Patterns in Children's Multisyllabic Word Productions. , 2001, Language, speech, and hearing services in schools.

[18]  Emily Butler.  Temporal Interpersonal Emotion Systems , 2011, Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc.

[19]  Samer Al Moubayed,et al.  Mole Madness - A Multi-Child, Fast-Paced, Speech-Controlled Game , 2015, AAAI Spring Symposia.

[20]  A. Harrist,et al.  Dyadic Synchrony: Its Structure and Function in Children's Development. , 2002 .

[21]  Mohamed Chetouani,et al.  Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines , 2012, IEEE Transactions on Affective Computing.