Teaching robots social autonomy from in situ human guidance

A robot was programmed to progressively learn appropriate social autonomous behavior from in situ human demonstrations and guidance. Striking the right balance between robot autonomy and human control is a core challenge in social robotics, in both technical and ethical terms. On the one hand, extended robot autonomy offers the potential for increased human productivity and for the off-loading of physical and cognitive tasks. On the other hand, making the most of human technical and social expertise, as well as maintaining accountability, is highly desirable. This is particularly relevant in domains such as medical therapy and education, where social robots hold substantial promise, but where there is a high cost to poorly performing autonomous systems, compounded by ethical concerns. We present a field study in which we evaluate SPARC (supervised progressively autonomous robot competencies), an innovative approach addressing this challenge whereby a robot progressively learns appropriate autonomous behavior from in situ human demonstrations and guidance. Using online machine learning techniques, we demonstrate that the robot could effectively acquire legible and congruent social policies in a high-dimensional child-tutoring situation needing only a limited number of demonstrations while preserving human supervision whenever desirable. By exploiting human expertise, our technique enables rapid learning of autonomous social and domain-specific policies in complex and nondeterministic environments. Last, we underline the generic properties of SPARC and discuss how this paradigm is relevant to a broad range of difficult human-robot interaction scenarios.

[1]  Tony Belpaeme,et al.  SPARC: Supervised Progressively Autonomous Robot Competencies , 2015, ICSR.

[2]  Manuela M. Veloso,et al.  Interactive Policy Learning through Confidence-Based Autonomy , 2014, J. Artif. Intell. Res..

[3]  Tony Belpaeme,et al.  Supervised autonomy for online learning in human-robot interaction , 2017, Pattern Recognit. Lett..

[4]  Laurel D. Riek,et al.  Wizard of Oz studies in HRI , 2012, J. Hum. Robot Interact..

[5]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[6]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[7]  Katherine K. Perkins,et al.  Factors promoting engaged exploration with computer simulations , 2010 .

[8]  Julie A. Shah,et al.  Apprenticeship Scheduling: Learning to Schedule from Human Experts , 2016, IJCAI.

[9]  Maya Cakmak,et al.  Power to the People: The Role of Humans in Interactive Machine Learning , 2014, AI Mag..

[10]  Gérard Bailly,et al.  Learning multimodal behavioral models for face-to-face social interaction , 2015, Journal on Multimodal User Interfaces.

[11]  Andrea Lockerd Thomaz,et al.  Teachable robots: Understanding human teaching behavior to build more effective robot learners , 2008, Artif. Intell..

[12]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[13]  David E. Meltzer,et al.  The relationship between mathematics preparation and conceptual learning gains in physics: A possible “hidden variable” in diagnostic pretest scores , 2002 .

[14]  Z. Dienes Bayesian Versus Orthodox Statistics: Which Side Are You On? , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[15]  H. Jeffreys The Theory of Probability , 1922 .

[16]  Brian Scassellati,et al.  Social robots for education: A review , 2018, Science Robotics.

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  Alan R. Wagner,et al.  Effect of Robot Performance on Human–Robot Trust in Time-Critical Situations , 2017, IEEE Transactions on Human-Machine Systems.

[19]  Ginevra Castellano,et al.  Discovering social interaction strategies for robots from restricted-perception Wizard-of-Oz studies , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[20]  Pierre Dillenbourg,et al.  Design for classroom orchestration , 2013, Comput. Educ..

[21]  Takayuki Kanda,et al.  Data-Driven HRI: Learning Social Behaviors by Example From Human–Human Interaction , 2016, IEEE Transactions on Robotics.

[22]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..