Quantifying teaching behavior in robot learning from demonstration

Learning from demonstration allows for rapid deployment of robot manipulators to a great many tasks, by relying on a person showing the robot what to do rather than programming it. While this approach provides many opportunities, measuring, evaluating, and improving the person’s teaching ability has remained largely unexplored in robot manipulation research. To this end, a model for learning from demonstration is presented here that incorporates the teacher’s understanding of, and influence on, the learner. The proposed model is used to clarify the teacher’s objectives during learning from demonstration, providing new views on how teaching failures and efficiency can be defined. The benefit of this approach is shown in two experiments ( n = 30 and n = 36 , respectively), which highlight the difficulty teachers have in providing effective demonstrations, and show how ~ 169 –180% improvement in teaching efficiency can be achieved through evaluation and feedback shaped by the proposed framework, relative to unguided teaching.

[1]  J. Gregory Trafton,et al.  Human modeling for human–robot collaboration , 2017, Int. J. Robotics Res..

[2]  Kevin Li,et al.  Evaluating Effects of User Experience and System Transparency on Trust in Automation , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[3]  Sungho Jo,et al.  Incremental Online Learning of Robot Behaviors From Selected Multiple Kinesthetic Teaching Trials , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[4]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[5]  S. Schaal.  Dynamic Movement Primitives -A Framework for Motor Control in Humans and Humanoid Robotics , 2006 .

[6]  Monica N. Nicolescu,et al.  Natural methods for robot task learning: instructive demonstrations, generalization and practice , 2003, AAMAS '03.

[7]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[8]  Brett Browning,et al.  Learning by demonstration with critique from a human teacher , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[9]  Alberto Montebelli,et al.  Incrementally assisted kinesthetic teaching for programming by demonstration , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[10]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[11]  Affan Pervez,et al.  Learning task-parameterized dynamic movement primitives using mixture of GMMs , 2018, Intell. Serv. Robotics.

[12]  Andrea Lockerd Thomaz,et al.  Robot Learning from Human Teachers , 2014, Robot Learning from Human Teachers.

[13]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[14]  Aude Billard,et al.  Task Parameterization Using Continuous Constraints Extracted From Human Demonstrations , 2015, IEEE Transactions on Robotics.

[15]  Thomas Zeugmann,et al.  Recent Developments in Algorithmic Teaching , 2009, LATA.

[16]  Tamim Asfour,et al.  Imitation Learning of Dual-Arm Manipulation Tasks in Humanoid Robots , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[17]  Thomas Hellström,et al.  A formalism for learning from demonstration , 2010, Paladyn J. Behav. Robotics.

[18]  Darwin G. Caldwell,et al.  Generalized Task-Parameterized Skill Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Xi Chen,et al.  Learning From Demonstration in the Wild , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[20]  Xiaojin Zhu,et al.  Machine Teaching: An Inverse Problem to Machine Learning and an Approach Toward Optimal Education , 2015, AAAI.

[21]  Emanuel Schmider,et al.  Is It Really Robust , 2010 .

[22]  Edward Grant,et al.  Learning for Control , 2019 .

[23]  M. Kearns,et al.  On the complexity of teaching , 1991, COLT '91.

[24]  Sylvain Calinon,et al.  A tutorial on task-parameterized movement learning and retrieval , 2016, Intell. Serv. Robotics.

[25]  Aude Billard,et al.  What is the Teacher"s Role in Robot Programming by Demonstration? - Toward Benchmarks for Improved Learning , 2007 .

[26]  Peng Zang,et al.  Batch versus interactive learning by demonstration , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[27]  A. Thomaz,et al.  Active Learning with Mixed Query Types in Learning from Demonstration , 2011 .

[28]  Jan Peters,et al.  Active Incremental Learning of Robot Movement Primitives , 2017, CoRL.

[29]  Sonia Chernova,et al.  Trajectory-Based Skill Learning Using Generalized Cylinders , 2018, Front. Robot. AI.

[30]  Katia Sycara,et al.  The role of trust in human-robot interaction , 2018 .

[31]  Pierre-Yves Oudeyer,et al.  Incremental local online Gaussian Mixture Regression for imitation learning of multiple tasks , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Siddhartha S. Srinivasa,et al.  Human-Robot Mutual Adaptation in Shared Autonomy , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[33]  Aude Billard,et al.  Incremental learning of gestures by imitation in a humanoid robot , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[34]  Fulvio Mastrogiovanni,et al.  Learning symbolic representations of actions from human demonstrations , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[35]  Carme Torras,et al.  Incremental Learning of Skills in a Task-Parameterized Gaussian Mixture Model , 2016, J. Intell. Robotic Syst..

[36]  Sonia Chernova,et al.  A Practical Comparison of Three Robot Learning from Demonstration Algorithm , 2012, International Journal of Social Robotics.

[37]  Aude Billard,et al.  Robot Learning from Failed Demonstrations , 2012, Int. J. Soc. Robotics.

[38]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[39]  Dominik Henrich,et al.  Robot programming by non-experts: Intuitiveness and robustness of One-Shot robot programming , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[40]  Yuchen Zhao,et al.  Teaching Human Teachers to Teach Robot Learners , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[41]  Dominik Henrich,et al.  One-shot robot programming by demonstration using an online oriented particles simulation , 2014, 2014 IEEE International Conference on Robotics and Biomimetics (ROBIO 2014).

[42]  Thomas Hellström,et al.  Understandable robots - What, Why, and How , 2018, Paladyn J. Behav. Robotics.

[43]  Pierre-Yves Oudeyer,et al.  A social learning formalism for learners trying to figure out what a teacher wants them to do , 2014, Paladyn J. Behav. Robotics.

[44]  Darwin G. Caldwell,et al.  Reinforcement Learning in Robotics: Applications and Real-World Challenges , 2013, Robotics.

[45]  Bilge Mutlu,et al.  How Do Humans Teach: On Curriculum Learning and Teaching Dimension , 2011, NIPS.

[46]  Maya Cakmak,et al.  Eliciting good teaching from humans for machine learners , 2014, Artif. Intell..

[47]  Manfred Tscheligi,et al.  Teaching a humanoid: A user study on learning by demonstration with HOAP-3 , 2009, RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication.

[48]  E. Erdfelder,et al.  Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses , 2009, Behavior research methods.