Fast Adaptation with Meta-Reinforcement Learning for Trust Modelling in Human-Robot Interaction

In socially assistive robotics, an important research area is the development of adaptation techniques and their effect on human-robot interaction. We present a meta-learning based policy gradient method for addressing the problem of adaptation in human-robot interaction and also investigate its role as a mechanism for trust modelling. By building an escape room scenario in mixed reality with a robot, we test our hypothesis that bi-directional trust can be influenced by different adaptation algorithms. We found that our proposed model increased the perceived trustworthiness of the robot and influenced the dynamics of gaining human’s trust. Additionally, participants evaluated that the robot perceived them as more trustworthy during the interactions with the meta-learning based adaptation compared to the previously studied statistical adaptation model.

[1]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Kristin E. Schaefer,et al.  Measuring Trust in Human Robot Interactions: Development of the “ Trust Perception Scale-HRI ” , 2016 .

[3]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[4]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[5]  Elizabeth J. Carter,et al.  Take One For the Team: The Effects of Error Severity in Collaborative Tasks with Social Robots , 2019, IVA.

[6]  Giulio Sandini,et al.  Trust and Social Engineering in Human Robot Interaction: Will a Robot Make You Disclose Sensitive Information, Conform to Its Recommendations or Gamble? , 2018, IEEE Robotics and Automation Letters.

[7]  Danica Kragic,et al.  A sensorimotor reinforcement learning framework for physical Human-Robot Interaction , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  Momotaz Begum,et al.  Deep Reinforcement Learning of Abstract Reasoning from Demonstrations , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[9]  Daniel J. McAllister.  Affect- and Cognition-Based Trust as Foundations for Interpersonal Cooperation in Organizations , 1995 .

[10]  Marcin Andrychowicz,et al.  One-Shot Imitation Learning , 2017, NIPS.

[11]  Thomas B. Sheridan,et al.  Human–Robot Interaction , 2016, Hum. Factors.

[12]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[13]  K. Dautenhahn,et al.  The Negative Attitudes Towards Robots Scale and reactions to robot behaviour in a live Human-Robot Interaction study , 2009 .

[14]  Philip A. Kragel,et al.  Decoding the Nature of Emotion in the Brain , 2016, Trends in Cognitive Sciences.

[15]  E. Hilgard.  The trilogy of mind: cognition, affection, and conation. , 1980, Journal of the history of the behavioral sciences.

[16]  Danica Kragic,et al.  A Comparison of Visualisation Methods for Disambiguating Verbal Requests in Human-Robot Interaction , 2018, 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[17]  S. Gosling,et al.  A very brief measure of the Big-Five personality domains , 2003 .

[18]  Joaquin Vanschoren,et al.  Meta-Learning: A Survey , 2018, Automated Machine Learning.

[19]  Stephen Marsh,et al.  Formalising Trust as a Computational Concept , 1994 .

[20]  Cynthia Breazeal,et al.  Computationally modeling interpersonal trust , 2013, Front. Psychol..

[21]  Ewart de Visser,et al.  Measurement of trust in human-robot collaboration , 2007, 2007 International Symposium on Collaborative Technologies and Systems.

[22]  Stevo Bozinovski,et al.  Emotion, Embodiment, and Consequence Driven Systems , 1996 .

[23]  Jessie Y. C. Chen,et al.  A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction , 2011, Hum. Factors.

[24]  Emily Mower Provost,et al.  Predicting the distribution of emotion perception: capturing inter-rater variability , 2017, ICMI.

[25]  Yuichiro Yoshikawa,et al.  Robot gains social intelligence through multimodal deep reinforcement learning , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[26]  Carman Neustaedter,et al.  Collaboration, Awareness, and Communication in Real-Life Escape Rooms , 2017, Conference on Designing Interactive Systems.

[27]  Cynthia Breazeal,et al.  Machine behaviour , 2019, Nature.

[28]  A. Evans,et al.  Survey and behavioral measurements of interpersonal trust , 2008 .

[29]  Susan G. Straus,et al.  All in due time: The development of trust in computer-mediated and face-to-face teams , 2006 .

[30]  Yuan Gao,et al.  When Robot Personalisation Does Not Help: Insights from a Robot-Supported Learning Study , 2018, 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[31]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[32]  Fillia Makedon,et al.  Task Engagement as Personalization Feedback for Socially-Assistive Robots and Cognitive Training , 2018 .

[33]  Peter L. Bartlett,et al.  RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning , 2016, ArXiv.

[34]  Omar Mubin,et al.  Emotion and Memory Model to Promote Mathematics Learning - An Exploratory Long-term Study , 2018, HAI.

[35]  Kristin E. Schaefer,et al.  The Perception And Measurement Of Human-robot Trust , 2013 .

[36]  K. Blomqvist.  The many faces of trust , 1997 .

[37]  J. H. Davis,et al.  An integrative model of organizational trust, Academy of Management Review, : . , 1995 .

[38]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[39]  Kerstin Dautenhahn,et al.  Would You Trust a (Faulty) Robot? Effects of Error, Task Type and Personality on Human-Robot Cooperation and Trust , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[40]  J. H. Davis,et al.  An Integrative Model Of Organizational Trust , 1995 .

[41]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[42]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[43]  Ana Paiva,et al.  Empathic Robots for Long-term Interaction , 2014, Int. J. Soc. Robotics.

[44]  Maja J. Mataric,et al.  Learning social behavior , 1997, Robotics Auton. Syst..