Nonverbal Robot Feedback for Human Teachers

Robots can learn preferences from human demonstrations, but their success depends on how informative these demonstrations are. Being informative is unfortunately very challenging, because during teaching, people typically get no transparency into what the robot already knows or has learned so far. In contrast, human students naturally provide a wealth of nonverbal feedback that reveals their level of understanding and engagement. In this work, we study how a robot can similarly provide feedback that is minimally disruptive, yet gives human teachers a better mental model of the robot learner, and thus enables them to teach more effectively. Our idea is that at any point, the robot can indicate what it thinks the correct next action is, shedding light on its current estimate of the human's preferences. We analyze how useful this feedback is, both in theory and with two user studies---one with a virtual character that tests the feedback itself, and one with a PR2 robot that uses gaze as the feedback mechanism. We find that feedback can be useful for improving both the quality of teaching and teachers' understanding of the robot's capability.

[1]  Michael Kearns,et al.  On the complexity of teaching , 1991, COLT '91.

[2]  Dean Pomerleau,et al.  Efficient Training of Artificial Neural Networks for Autonomous Navigation , 1991, Neural Computation.

[3]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[4]  Thomas A. Angelo,et al.  Classroom Assessment Techniques: A Handbook for College Teachers. Second Edition. , 1993 .

[5]  M. Nowak,et al.  A strategy of win-stay, lose-shift that outperforms tit-for-tat in the Prisoner's Dilemma game , 1993, Nature.

[6]  J. M. Webb,et al.  Influence of Pedagogical Expertise and Feedback on Assessing Student Comprehension From Nonverbal Behavior , 1997 .

[7]  Luc Steels,et al.  Aibo''s first words. the social learning of language and meaning. Evolution of Communication , 2002 .

[8]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[9]  Tetsunari Inamura Masayuki Inaba Hirochika.  Acquisition of Probabilistic Behavior Decision Model based on the Interactive Teaching Method , 2001 .

[10]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[11]  Yoshiaki Shirai,et al.  Interactive Teaching of a Mobile Robot , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[12]  Rajesh P. N. Rao,et al.  Active Imitation Learning , 2007, AAAI.

[13]  Aude Billard,et al.  Active Teaching in Robot Programming by Demonstration , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[14]  Eyal Amir,et al.  Bayesian Inverse Reinforcement Learning , 2007, IJCAI.

[15]  Andrea Lockerd Thomaz,et al.  Teachable robots: Understanding human teaching behavior to build more effective robot learners , 2008, Artif. Intell..

[16]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[17]  Chris L. Baker,et al.  Action understanding as inverse planning , 2009, Cognition.

[18]  Thomas Zeugmann,et al.  Recent Developments in Algorithmic Teaching , 2009, LATA.

[19]  Manuela M. Veloso,et al.  Interactive Policy Learning through Confidence-Based Autonomy , 2014, J. Artif. Intell. Res..

[20]  A. Thomaz,et al.  Transparent active learning for robots , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[21]  Maya Cakmak,et al.  Optimality of human teachers for robot learners , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[22]  Maya Cakmak,et al.  Designing Interactions for Robot Active Learners , 2010, IEEE Transactions on Autonomous Mental Development.

[23]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[24]  Bilge Mutlu,et al.  How Do Humans Teach: On Curriculum Learning and Teaching Dimension , 2011, NIPS.

[25]  David Silver,et al.  Active learning from demonstration for robust autonomous navigation , 2012, 2012 IEEE International Conference on Robotics and Automation.

[26]  Maya Cakmak,et al.  Designing robot learners that ask good questions , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[27]  Peter Ford Dominey,et al.  I Reach Faster When I See You Look: Gaze Effects in Human–Human and Human–Robot Face-to-Face Cooperation , 2012, Front. Neurorobot..

[28]  Sergey Levine,et al.  Continuous Inverse Optimal Control with Locally Optimal Examples , 2012, ICML.

[29]  Karon E. MacLean,et al.  Meet Me where I’m Gazing: How Shared Attention Gaze Affects Human-Robot Handover Timing , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[30]  Gi Hyun Lim,et al.  Interactive teaching and experience extraction for learning about objects and robot activities , 2014, The 23rd IEEE International Symposium on Robot and Human Interactive Communication.

[31]  Siddhartha S. Srinivasa,et al.  Deliberate Delays During Robot-to-Human Handovers Improve Compliance With Gaze Communication , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[32]  Xiaojin Zhu,et al.  Machine Teaching: An Inverse Problem to Machine Learning and an Approach Toward Optimal Education , 2015, AAAI.

[33]  Joshua B. Tenenbaum,et al.  The Naïve Utility Calculus: Computational Principles Underlying Commonsense Psychology , 2016, Trends in Cognitive Sciences.

[34]  Brian Scassellati,et al.  Modeling communicative behaviors for object references in human-robot interaction , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[35]  Alan A. Stocker,et al.  Human Decision-Making under Limited Time , 2016, NIPS.

[36]  David Whitney,et al.  Reducing errors in object-fetching interactions through social feedback , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[37]  Julian Hough,et al.  It's Not What You Do, It's How You Do It: Grounding Uncertainty for a Simple Robot , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[38]  Anca D. Dragan,et al.  Expressive Robot Motion Timing , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[39]  B. Scassellati,et al.  Social eye gaze in human-robot interaction , 2017, J. Hum. Robot Interact..

[40]  Yuchen Zhao,et al.  Teaching Human Teachers to Teach Robot Learners , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).