Expressive Cognitive Architecture for a Curious Social Robot

Artificial curiosity, based on developmental psychology concepts wherein an agent attempts to maximize its learning progress, has gained much attention in recent years. Similarly, social robots are slowly integrating into our daily lives, in schools, factories, and in our homes. In this contribution, we integrate recent advances in artificial curiosity and social robots into a single expressive cognitive architecture. It is composed of artificial curiosity and social expressivity modules and their unique link, i.e., the robot verbally and non-verbally communicates its internally estimated learning progress, or learnability, to its human companion. We implemented this architecture in an interaction where a fully autonomous robot took turns with a child trying to select and solve tangram puzzles on a tablet. During the curious robot’s turn, it selected its estimated most learnable tangram to play, communicated its selection to the child, and then attempted at solving it. We validated the implemented architecture and showed that the robot learned, estimated its learnability, and improved when its selection was based on its learnability estimation. Moreover, we ran a comparison study between curious and non-curious robots, and showed that the robot’s curiosity-based behavior influenced the child’s selections. Based on the artificial curiosity module of the robot, we have formulated an equation that estimates each child’s moment-by-moment curiosity based on their selections. This analysis revealed an overall significant decrease in estimated curiosity during the interaction. However, this drop in estimated curiosity was significantly larger with the non-curious robot, compared to the curious one. These results suggest that the new architecture is a promising new approach to integrate state-of-the-art curiosity-based algorithms to the growing field of social robots.

[1]  Mirko Gelsomini,et al.  Telling Stories to Robots: The Effect of Backchanneling on a Child's Storytelling * , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[2]  C. Dweck,et al.  Mindsets and human nature: promoting change in the Middle East, the schoolyard, the racial divide, and willpower. , 2012, The American psychologist.

[3]  S. Shankar Sastry,et al.  Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning , 2017, ArXiv.

[4]  Kemal Oflazer Solving tangram puzzles: A connectionist approach , 1993, Int. J. Intell. Syst..

[5]  Cynthia Breazeal,et al.  Designing sociable robots , 2002 .

[6]  R. Endsley,et al.  Answering Young Children's Questions as a Determinant of Their Subsequent Question-Asking Behavior. , 1975 .

[7]  Dana Kulic,et al.  Expression of Curiosity in Social Robots: Design, Perception, and Effects on Behaviour , 2019, CHI.

[8]  Ruth Campbell,et al.  Age, gender, and puberty influence the development of facial emotion recognition , 2015, Front. Psychol..

[9]  Marijn F. Stollenga,et al.  Continual curiosity-driven skill acquisition from high-dimensional video inputs for humanoid robots , 2017, Artif. Intell..

[10]  Cynthia Breazeal,et al.  Can Children Catch Curiosity from a Social Robot? , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[11]  Laurenz Wiskott,et al.  Intrinsically Motivated Acquisition of Modular Slow Features for Humanoids in Continuous and Non-Stationary Environments , 2017, ArXiv.

[12]  Paul J Silvia,et al.  The Latent Structure of Trait Curiosity: Evidence for Interest and Deprivation Curiosity Dimensions , 2006, Journal of personality assessment.

[13]  Jonas Beskow,et al.  Reverse Engineering Psychologically Valid Facial Expressions of Emotion into Social Robots , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[14]  S. Engel The Hungry Mind: The Origins of Curiosity in Childhood , 2015 .

[15]  A. Barto,et al.  Intrinsic motivations and open-ended development in animals, humans, and robots: an overview , 2014, Front. Psychol..

[16]  Jeff Clune,et al.  Deep Curiosity Search: Intra-Life Exploration Improves Performance on Challenging Deep Reinforcement Learning Problems , 2018, ArXiv.

[17]  Rhodri Cusack,et al.  Online recruitment and testing of infants with Mechanical Turk. , 2017, Journal of experimental child psychology.

[18]  Cynthia Breazeal,et al.  Growing Growth Mindset with a Social Robot Peer , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[19]  Stefan Kopp,et al.  To Err is Human(-like): Effects of Robot Gesture on Perceived Anthropomorphism and Likability , 2013, International Journal of Social Robotics.

[20]  M. Lepper,et al.  Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. , 1996 .

[21]  M. Kang Three experimental studies of reward and decision making , 2010 .

[22]  Jamie J. Jirout,et al.  Curiosity and the Development of Question Generation Skills , 2011, AAAI Fall Symposium: Question Generation.

[23]  P. Aguilar,et al.  Evaluación de proceso de la Intervención Relacional basada en el apego a través del análisis de dos casos , 2020 .

[24]  J. Weisz,et al.  Using Mechanical Turk to Study Family Processes and Youth Mental Health: A Test of Feasibility , 2015, Journal of Child and Family Studies.

[25]  Philippe G. Schyns,et al.  Equipping social robots with culturally-sensitive facial expressions of emotion using data-driven methods , 2019, 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019).

[26]  Nuttapong Chentanez,et al.  Intrinsically Motivated Learning of Hierarchical Collections of Skills , 2004 .

[27]  Yuichiro Yoshikawa,et al.  Intrinsically motivated reinforcement learning for human-robot interaction in the real-world , 2018, Neural Networks.

[28]  Tony Belpaeme,et al.  The Impact of Robot Tutor Nonverbal Social Behavior on Child Learning , 2017, Front. ICT.

[29]  Goren Gordon,et al.  Priming, enabling and assessment of curiosity , 2019, Educational Technology Research and Development.

[30]  C. Breazeal Role of expressive behaviour for robots that learn from people , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[31]  Wafa Johal,et al.  Social Human-Robot Interaction: A New Cognitive and Affective Interaction-Oriented Architecture , 2016, ICSR.

[32]  Norihiro Hagita,et al.  Lexical entrainment in human-robot interaction: Can robots entrain human vocabulary? , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Ana Paiva,et al.  How Facial Expressions and Small Talk May Influence Trust in a Robot , 2016, ICSR.

[34]  Goren Gordon Social behaviour as an emergent property of embodied curiosity: a robotics perspective , 2019, Philosophical Transactions of the Royal Society B.

[35]  Kenneth O. Stanley,et al.  Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents , 2017, NeurIPS.

[36]  Cynthia Breazeal,et al.  Detecting the Trustworthiness of Novel Partners in Economic Exchange , 2012, Psychological science.

[37]  John K. Tsotsos,et al.  A Review of 40 Years of Cognitive Architecture Research: Focus on Perception, Attention, Learning and Applications , 2016, ArXiv.

[38]  Jürgen Schmidhuber,et al.  A possibility for implementing curiosity and boredom in model-building neural controllers , 1991 .

[39]  Sandy H. Huang,et al.  Learning Gentle Object Manipulation with Curiosity-Driven Deep Reinforcement Learning , 2019, ArXiv.

[40]  S. Engel,et al.  The Hungry Mind: The Origins of Curiosity in Childhood , 2015 .

[41]  Alexander Wendt,et al.  A psychoanalytically-inspired motivational and emotional system for autonomous agents , 2013, IECON 2013 - 39th Annual Conference of the IEEE Industrial Electronics Society.

[42]  Brian Scassellati,et al.  How to build robots that make friends and influence people , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[43]  Pierre-Yves Oudeyer,et al.  What is Intrinsic Motivation? A Typology of Computational Approaches , 2007, Frontiers Neurorobotics.

[44]  G. Loewenstein The psychology of curiosity: A review and reinterpretation. , 1994 .

[45]  Pierre-Yves Oudeyer,et al.  Intrinsic motivation, curiosity, and learning: Theory and applications in educational technologies. , 2016, Progress in brain research.

[46]  Nicole C. Krämer,et al.  The Effects of Humanlike and Robot-Specific Affective Nonverbal Behavior on Perception, Emotion, and Behavior , 2018, International Journal of Social Robotics.

[47]  Goren Gordon,et al.  Social Robots as Physical Curiosity Assessment Tools , 2018, 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[48]  Stefan Wermter,et al.  Deep intrinsically motivated continuous actor-critic for efficient robotic visuomotor skill learning , 2018, Paladyn J. Behav. Robotics.

[49]  Alexei A. Efros,et al.  Curiosity-Driven Exploration by Self-Supervised Prediction , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[50]  Paul Baxter,et al.  Cognitive architecture for human–robot interaction: Towards behavioural alignment , 2013, BICA 2013.

[51]  Julian Togelius,et al.  Deep Learning for Video Game Playing , 2017, IEEE Transactions on Games.

[52]  D. Klahr,et al.  Children's Scientific Curiosity: In Search of an Operational Definition of an Elusive Concept. , 2012 .

[53]  Alex Mihailidis,et al.  Social Robots and Seniors: A Comparative Study on the Influence of Dynamic Social Features on Human–Robot Interaction , 2018, International Journal of Social Robotics.

[54]  Diana Löffler,et al.  Multimodal Expression of Artificial Emotion in Social Robots Using Color, Motion and Sound , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[55]  S. Hunnius,et al.  Modulating mimicry: Exploring the roles of inhibitory control and social understanding in 5-year-olds' behavioral mimicry , 2018, PloS one.

[56]  Matthias J. Gruber,et al.  States of Curiosity Modulate Hippocampus-Dependent Learning via the Dopaminergic Circuit , 2014, Neuron.

[57]  A. Acquisti,et al.  Reputation as a sufficient condition for data quality on Amazon Mechanical Turk , 2013, Behavior Research Methods.

[58]  Goren Gordon,et al.  Digital Interactive Quantitative Curiosity Assessment Tool: Questions Worlds , 2020 .

[59]  Pierre-Yves Oudeyer,et al.  CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning , 2018, ICML.

[60]  Peter Seitz,et al.  Minimum class entropy: A maximum information approach to layered networks , 1989, Neural Networks.

[61]  S. Hunnius,et al.  Little chameleons: The development of social mimicry during early childhood. , 2016, Journal of experimental child psychology.

[62]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[63]  Joshua B. Tenenbaum,et al.  Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation , 2016, NIPS.

[64]  Stefan Kopp,et al.  The Effect of a Robot’s Gestures and Adaptive Tutoring on Children’s Acquisition of Second Language Vocabularies , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[65]  Goren Gordon,et al.  Infant-inspired intrinsically motivated curious robots , 2020, Current Opinion in Behavioral Sciences.

[66]  Frank D. Naylor,et al.  A State‐Trait Curiosity Inventory , 1981 .