Evaluating the Engagement with Social Robots

To interact and cooperate with humans in their daily-life activities, robots should exhibit human-like “intelligence”. This skill will substantially emerge from the interconnection of all the algorithms used to ensure cognitive and interaction capabilities. While new robotics technologies allow us to extend such abilities, their evaluation for social interaction is still challenging. The quality of a human–robot interaction can not be reduced to the evaluation of the employed algorithms: we should integrate the engagement information that naturally arises during interaction in response to the robot’s behaviors. In this paper we want to show a practical approach to evaluate the engagement aroused during interactions between humans and social robots. We will introduce a set of metrics useful in direct, face to face scenarios, based on the behaviors analysis of the human partners. We will show how such metrics are useful to assess how the robot is perceived by humans and how this perception changes according to the behaviors shown by the social robot. We discuss experimental results obtained in two human-interaction studies, with the robots Nao and iCub respectively.

[1]  John F. Kihlstrom,et al.  Personality and social intelligence , 1987 .

[2]  R. Nickerson Confirmation Bias: A Ubiquitous Phenomenon in Many Guises , 1998 .

[3]  Kerstin Dautenhahn,et al.  Getting to know each other - Artificial social intelligence for autonomous robots , 1995, Robotics Auton. Syst..

[4]  M. Chetouani,et al.  How children with autism spectrum disorder behave and explore the 4-dimensional (spatial 3D + time) environment during a joint attention induction task with a robot , 2014 .

[5]  Philippe Gaussier,et al.  Using the Rhythm of Nonverbal Human–Robot Interaction as a Signal for Learning , 2011, IEEE Transactions on Autonomous Mental Development.

[6]  BlakeAndrew,et al.  Real-time human pose recognition in parts from single depth images , 2013 .

[7]  Hiroshi Ishiguro,et al.  Interactive humanoids and androids as ideal interfaces for humans , 2005, ICMI '05.

[8]  Candace L. Sidner,et al.  Where to look: a study of human-robot engagement , 2004, IUI '04.

[9]  Andrea Lockerd Thomaz,et al.  Effects of nonverbal communication on efficiency and robustness in human-robot teamwork , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Sofiane Boucenna,et al.  Learning of Social Signatures Through Imitation Game Between a Robot and a Human Partner , 2014, IEEE Transactions on Autonomous Mental Development.

[11]  Kerstin Dautenhahn,et al.  Socially intelligent robots: dimensions of human–robot interaction , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[12]  Noël Crespi,et al.  QoE Aware Service Delivery in Distributed Environment , 2011, 2011 IEEE Workshops of International Conference on Advanced Information Networking and Applications.

[13]  Elaine Toms,et al.  What is user engagement? A conceptual framework for defining user engagement with technology , 2008, J. Assoc. Inf. Sci. Technol..

[14]  Tatsuo Arai,et al.  Development of a scale of perception to humanoid robots: PERNOD , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Mohamed Chetouani,et al.  Automatic measure of imitation during social interaction: A behavioral and hyperscanning-EEG benchmark , 2015, Pattern Recognit. Lett..

[16]  Mohamed Chetouani,et al.  Tracking Posture and Head Movements of Impaired People During Interactions with Robots , 2013, ICIAP Workshops.

[17]  Candace L. Sidner,et al.  Recognizing engagement in human-robot interaction , 2010, HRI 2010.

[18]  Dirk Heylen,et al.  Bridging the Gap between Social Animal and Unsocial Machine: A Survey of Social Signal Processing , 2012, IEEE Transactions on Affective Computing.

[19]  Hiroshi Ishiguro,et al.  Android Science , 2005, ISRR.

[20]  Mohamed Chetouani,et al.  Robot initiative in a team learning task increases the rhythm of interaction but not the perceived engagement , 2014, Front. Neurorobot..

[21]  P. Ekman,et al.  The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding , 1969 .

[22]  Mohamed Chetouani,et al.  Multimodal People Engagement with iCub , 2012, BICA.

[23]  Pierre Dillenbourg,et al.  The Dynamics of Anthropomorphism in Robotics , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[24]  Alexander Raake,et al.  Quality and Quality of Experience , 2014, Quality of Experience.

[25]  Illah R. Nourbakhsh,et al.  A survey of socially interactive robots , 2003, Robotics Auton. Syst..

[26]  Bernard C. K. Choi,et al.  A Catalog of Biases in Questionnaires , 2004, Preventing chronic disease.

[27]  Thierry Baccino,et al.  A Computational Cognitive Model of Information Search in Textual Materials , 2012, Cognitive Computation.

[28]  Giulio Sandini,et al.  The iCub Platform: A Tool for Studying Intrinsically Motivated Learning , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[29]  Brett Browning,et al.  Teacher feedback to scaffold and refine demonstrated motion primitives on a mobile robot , 2011, Robotics Auton. Syst..

[30]  Ana Paiva,et al.  Automatic analysis of affective postures and body motion to detect engagement with a game companion , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[31]  Brian Scassellati,et al.  No fair!! An interaction with a cheating robot , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[32]  Sofiane Boucenna,et al.  Interactive Technologies for Autistic Children: A Review , 2014, Cognitive Computation.

[33]  P. H. Miller Theories of developmental psychology , 1983 .

[34]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[35]  John F. Kihlstrom,et al.  Handbook of Intelligence: Social Intelligence , 2000 .

[36]  Anthony G. Pipe,et al.  Perception of own and robot engagement in human-robot interactions and their dependence on robotics knowledge , 2014, Robotics Auton. Syst..

[37]  Karen Cortell Reisman "The art of asking questions". , 2008, Texas dental journal.

[38]  Maja J. Mataric,et al.  Investigating Implicit Cues for User State Estimation in Human-Robot Interaction Using Physiological Measurements , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[39]  G. Katona,et al.  The Art of Asking Questions , 1951 .

[40]  Shanyang Zhao,et al.  Toward a Taxonomy of Copresence , 2003, Presence: Teleoperators & Virtual Environments.

[41]  Matthias Scheutz,et al.  Incremental natural language processing for HRI , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[42]  Maja Pantic,et al.  Social signal processing: Survey of an emerging domain , 2009, Image Vis. Comput..

[43]  Alois Knoll,et al.  The roles of haptic-ostensive referring expressions in cooperative, task-based human-robot dialogue , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[44]  Philippe Gaussier,et al.  A Robot Learns the Facial Expressions Recognition and Face/Non-face Discrimination Through an Imitation Game , 2014, Int. J. Soc. Robotics.

[45]  Pierre-Yves Oudeyer,et al.  Object Learning Through Active Exploration , 2014, IEEE Transactions on Autonomous Mental Development.

[46]  M. Tomasello Joint attention as social cognition. , 1995 .

[47]  Tatsuo Arai,et al.  New measurement of psychological safety for humanoid , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[48]  Mohamed Chetouani,et al.  Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines , 2012, IEEE Transactions on Affective Computing.

[49]  D. Goleman,et al.  Social Intelligence , 2006 .

[50]  Kerstin Fischer,et al.  The impact of the contingency of robot feedback on HRI , 2013, 2013 International Conference on Collaboration Technologies and Systems (CTS).

[51]  Sukhvinder S. Obhi,et al.  Moving together: toward understanding the mechanisms of joint action , 2011, Experimental Brain Research.

[52]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[53]  A. Furnham Response bias, social desirability and dissimulation , 1986 .

[54]  Brian Scassellati,et al.  Quantitative metrics of social response for autism diagnosis , 2005, ROMAN 2005. IEEE International Workshop on Robot and Human Interactive Communication, 2005..

[55]  P. Vernon,et al.  Some Characteristics of the Good Judge of Personality , 1933 .

[56]  Paulo Menezes,et al.  Face tracking and hand gesture recognition for human-robot interaction , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[57]  Patricia Shaw,et al.  A psychology based approach for longitudinal development in cognitive robotics , 2014, Front. Neurorobot..

[58]  Kay Connelly,et al.  Toward total quality of experience: A QoE model in a communication ecosystem , 2012, IEEE Communications Magazine.

[59]  Candace L. Sidner,et al.  Explorations in engagement for humans and robots , 2005, Artif. Intell..

[60]  Cynthia Breazeal,et al.  Toward sociable robots , 2003, Robotics Auton. Syst..

[61]  Georgios N. Yannakakis,et al.  Entertainment capture through heart rate activity in physical interactive playgrounds , 2008, User Modeling and User-Adapted Interaction.

[62]  Cynthia Breazeal,et al.  Designing sociable robots , 2002 .

[63]  Emanuele Frontoni,et al.  Customers’ activity recognition in intelligent retail environments , 2013 .

[64]  Siddhartha S. Srinivasa,et al.  Deliberate Delays During Robot-to-Human Handovers Improve Compliance With Gaze Communication , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[65]  M. Tomasello,et al.  Joint attention and early language. , 1986, Child development.

[66]  Dana Kulic,et al.  Affective State Estimation for Human–Robot Interaction , 2007, IEEE Transactions on Robotics.

[67]  Trevor Darrell,et al.  Nodding in conversations with a robot , 2004, CHI EA '04.

[68]  Matthias Scheutz,et al.  Robust spoken instruction understanding for HRI , 2010, HRI 2010.

[69]  Aaron Steinfeld,et al.  A deceptive robot referee in a multiplayer gaming environment , 2011, 2011 International Conference on Collaboration Technologies and Systems (CTS).

[70]  Brian Scassellati,et al.  How Social Robots Will Help Us to Diagnose, Treat, and Understand Autism , 2005, ISRR.

[71]  Matteo Munaro,et al.  A distributed perception infrastructure for robot assisted living , 2014, Robotics Auton. Syst..

[72]  Sharon L. Oviatt,et al.  Designing speech and language interactions , 2014, CHI Extended Abstracts.

[73]  Alexander I. Rudnicky,et al.  Heterogeneous Multi-Robot Dialogues for Search Tasks , 2005 .

[74]  Mohamed Chetouani,et al.  Oxytocin shapes parental motion during father–infant interaction , 2013, Biology Letters.

[75]  E. Thorndike Intelligence and its uses. , 1920 .

[76]  Emanuele Menegatti,et al.  A Multimodal Distributed Intelligent Environment for a Safer Home , 2012, IAS.

[77]  Brian Scassellati,et al.  Socially assistive robotics [Grand Challenges of Robotics] , 2007, IEEE Robotics & Automation Magazine.

[78]  Marco Mirolli,et al.  Intrinsically Motivated Learning in Natural and Artificial Systems , 2013 .

[79]  Francesco Mondada,et al.  Lessons learned from robotic vacuum cleaners entering the home ecosystem , 2014, Robotics Auton. Syst..

[80]  Justin W. Hart,et al.  No fair!!: an interaction with a cheating robot , 2010, HRI 2010.

[81]  C. Breazeal Sociable Machines: Expressive Social Ex-change Between Humans and Robots , 2000 .

[82]  Timothy F. Cootes,et al.  Feature Detection and Tracking with Constrained Local Models , 2006, BMVC.

[83]  Andrea Lockerd Thomaz,et al.  Multi-cue Contingency Detection , 2012, Int. J. Soc. Robotics.

[84]  Bruce Edmonds,et al.  Social Intelligence , 1999, Comput. Math. Organ. Theory.

[85]  F. Kaplan,et al.  The challenges of joint attention , 2006 .

[86]  S. Baron-Cohen Mindblindness: An Essay on Autism and Theory of Mind , 1997 .

[87]  T. Nishida,et al.  Combining Multiple Types of Eye-gaze Information to Predict User ’ s Conversational Engagement , 2011 .