Multimodal child-robot interaction: building social bonds

For robots to interact effectively with human users they must be capable of coordinated, timely behavior in response to social context. The Adaptive Strategies for Sustainable Long-Term Social Interaction (ALIZ-E) project focuses on the design of long-term, adaptive social interaction between robots and child users in real-world settings. In this paper, we report on the iterative approach taken to scientific and technical developments toward this goal: advancing individual technical competencies and integrating them to form an autonomous robotic system for evaluation "in the wild." The first evaluation iterations have shown the potential of this methodology in terms of adaptation of the robot to the interactant and the resulting influences on engagement. This sets the foundation for an ongoing research program that seeks to develop technologies for social robot companions.

[1]  Koen V. Hindriks,et al.  The icat as a natural interaction partner: playing go fish with a robot , 2011, AAMAS'11.

[2]  Michael J. Beatty,et al.  Is there empirical evidence for a nonverbal profile of extraversion?: a meta‐analysis and critique of the literature , 2004 .

[3]  Paul Edward Baxter.  Foundations of a Constructivist Memory-Based approach to Cognitive Robotics , 2010 .

[4]  Fabio Tesser,et al.  An Event-Based Conversational System for the Nao Robot , 2011, IWSDS.

[5]  Steve J. Young,et al.  Partially observable Markov decision processes for spoken dialog systems , 2007, Comput. Speech Lang..

[6]  Aryel Beck,et al.  Towards an Affect Space for robots to display emotional body language , 2010, 19th International Symposium in Robot and Human Interactive Communication.

[7]  Angelo Cangelosi,et al.  Epigenetic Robotics Architecture (ERA) , 2010, IEEE Transactions on Autonomous Mental Development.

[8]  Tony Belpaeme,et al.  A touchscreen-based ‘Sandtray’ to facilitate, mediate and contextualise human-robot social interaction , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[9]  J. Gregory Trafton,et al.  Integrating cognition, perception and action through mental simulation in robots , 2004, Robotics Auton. Syst..

[10]  Peter Muris,et al.  Psychometric properties of the Big Five Questionnaire for Children (BFQ-C) in a Dutch sample of young adolescents , 2005 .

[11]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: part 1.: an account of basic findings , 1988 .

[12]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination in Human Behavior , 1975, Perspectives in Social Psychology.

[13]  Tony Belpaeme,et al.  From Penguins to Parakeets : a Developmental Approach to Modelling Conceptual Prototypes , 2012 .

[14]  Mark Steedman,et al.  Information Structure and the Syntax-Phonology Interface , 2000, Linguistic Inquiry.

[15]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[16]  Hiromitsu Hattori,et al.  Advanced Agent Technology , 2011, Lecture Notes in Computer Science.

[17]  Hichem Sahli,et al.  Relevance Vector Machine Based Speech Emotion Recognition , 2011, ACII.

[18]  Ivana Kruijff-Korbayová,et al.  Towards augmenting dialogue strategy management with multimodal sub-symbolic context , 2012 .

[19]  Heiga Zen,et al.  Statistical Parametric Speech Synthesis , 2007, IEEE International Conference on Acoustics, Speech, and Signal Processing.

[20]  Tony Belpaeme,et al.  How to use non-linguistic utterances to convey emotion in child-robot interaction , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[21]  Tony Belpaeme,et al.  “And what is a Seasnake?” Modelling the acquisition of concept prototypes in a developmental framework , 2012, 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[22]  Sathish Pammi.  Prosody control in HMM-based speech synthesis , 2011 .

[23]  Philippe Gaussier,et al.  Using the Interaction Rhythm as a Natural Reinforcement Signal for Social Robots: A Matter of Belief , 2010, ICSR.

[24]  Hichem Sahli,et al.  Real-Time Dance Pattern Recognition Invariant to Anthropometric and Temporal Differences , 2012, ACIVS.

[25]  Tetsuo Tomiyama,et al.  Advanced Engineering Informatics , 2007, Adv. Eng. Informatics.

[26]  Paul Baxter,et al.  Child-robot interaction in the wild: advice to the aspiring experimenter , 2011, ICMI '11.

[27]  Hichem Sahli,et al.  Context-Independent Facial Action Unit Recognition Using Shape and Gabor Phase Information , 2011, ACII.

[28]  J. Fuster.  Network memory , 1997, Trends in Neurosciences.

[29]  Jason Baldridge,et al.  Multi-Modal Combinatory Categorial Grammar , 2003, EACL.

[30]  Werner Verhelst,et al.  On Noise Robust Voice Activity Detection , 2011, INTERSPEECH.

[31]  Oliver Lemon,et al.  Recent research advances in Reinforcement Learning in Spoken Dialogue Systems , 2009, The Knowledge Engineering Review.

[32]  François Michaud,et al.  Going into the wild in child–robot interaction studies: issues in social robotic development , 2008, Intell. Serv. Robotics.

[33]  Tony Belpaeme,et al.  A review of long-term memory in natural and synthetic systems , 2012, Adapt. Behav..

[34]  Milica Gasic,et al.  The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management , 2010, Comput. Speech Lang..

[35]  Mark Steedman,et al.  The syntactic process , 2004, Language, speech, and communication.

[36]  GerosaMatteo,et al.  Acoustic variability and automatic recognition of children's speech , 2007 .

[37]  Blaise Roger Marie Thomson,et al.  Statistical methods for spoken dialogue management , 2013 .

[38]  R A Johnston,et al.  Understanding face recognition with an interactive activation model. , 1990, British journal of psychology.

[39]  Ivana Kruijff-Korbayová,et al.  Children's adaptation in multi-session interaction with a humanoid robot , 2012, 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication.

[40]  Fumihide Tanaka,et al.  Socialization between toddlers and robots at an early childhood education center , 2007, Proceedings of the National Academy of Sciences.

[41]  Cynthia Breazeal,et al.  Emotion and sociable humanoid robots , 2003, Int. J. Hum. Comput. Stud..

[42]  Joris B. Janssen,et al.  Motivating Children to Learn Arithmetic with an Adaptive Robot Game , 2011, ICSR.

[43]  Koen V. Hindriks,et al.  Programming Rational Agents in GOAL , 2009, Multi-Agent Programming, Languages, Tools and Applications.

[44]  Pierre Blazevic,et al.  The NAO humanoid: a combination of performance and affordability , 2008, ArXiv.

[45]  M. Bar.  The proactive brain: using analogies and associations to generate predictions , 2007, Trends in Cognitive Sciences.

[46]  Nick Hawes,et al.  Engineering intelligent information-processing systems with CAST , 2010, Adv. Eng. Informatics.

[47]  Jason Baldridge,et al.  Lexically specified derivational control in combinatory categorial grammar , 2002 .

[48]  Tony Belpaeme,et al.  Memory-Centred Architectures: Perspectives on Human-Level Cognitive Competencies , 2011, AAAI Fall Symposium: Advances in Cognitive Systems.

[49]  Keiichi Tokuda,et al.  Speech synthesis using HMMs with dynamic features , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[50]  Marc Schröder,et al.  Open Source Voice Creation Toolkit for the MARY TTS Platform , 2011, INTERSPEECH.

[51]  Fabio Brugnara,et al.  Acoustic variability and automatic recognition of children's speech , 2007, Speech Commun..

[52]  Takayuki Kanda,et al.  Interactive Robots as Social Partners and Peer Tutors for Children: A Field Trial , 2004, Hum. Comput. Interact..

[53]  Nina Dethlefs,et al.  Optimizing Situated Dialogue Management in Unknown Environments , 2011, INTERSPEECH.

[54]  Fabio Tesser,et al.  Children Interpretation of Emotional Body Language Displayed by a Robot , 2011, ICSR.

[55]  WoodRachel,et al.  Multimodal child-robot interaction , 2013, HRI 2013.

[56]  A. Fine.  Handbook on animal-assisted therapy. Theoretical foundations and guidelines for practice , 2010 .

[57]  Joelle Pineau,et al.  Spoken Dialogue Management Using Probabilistic Reasoning , 2000, ACL.

[58]  T W Draper,et al.  Using a personal robot to teach young children. , 1992, The Journal of genetic psychology.

[59]  Marc Schröder,et al.  The German Text-to-Speech Synthesis System MARY: A Tool for Research, Development and Teaching , 2003, Int. J. Speech Technol..

[60]  Brett Stevens,et al.  Emotional body language displayed by artificial agents , 2012, TIIS.