Teacher-Aware Active Robot Learning

This paper investigates Active Robot Learning strategies that take into account the effort of the user in an interactive learning scenario. Most research claims that Active Learning's sample efficiency can reduce training time and therefore the effort of the human teacher. We argue that the performance driven query selection of standard Active Learning can make the job of the human teacher difficult, resulting in a decrease in training quality due to slowdowns or increased error rates. We investigate this issue by proposing a learning strategy that aims to minimize the user's workload by taking into account the flow of the questions. We compare this strategy against a standard Active Learning strategy based on uncertainty sampling and a third strategy being an hybrid of the two. After studying in simulation the validity and the behavior of these approaches, we conducted a user study where 26 subjects interacted with a NAO robot embodying the presented strategies. We reports results from both the robot's performance and the human teacher's perspectives, observing how the hybrid strategy represents a good compromise between learning performance and user's experienced workload. Based on the results, we provide recommendations on the development of Active Robot Learning strategies going beyond robot's performance.

[1]  Edwin Lughofer,et al.  Hybrid active learning for reducing the annotation effort of operators in classification systems , 2012, Pattern Recognit..

[2]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[3]  María Malfaz,et al.  Analyzing the Impact of Different Feature Queries in Active Learning for Social Robots , 2018, Int. J. Soc. Robotics.

[4]  Maya Cakmak,et al.  Designing robot learners that ask good questions , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[5]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[6]  Rainer Stiefelhagen,et al.  Recovering the Missing Link: Predicting Class-Attribute Associations for Unsupervised Zero-Shot Learning , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Rohae Myung,et al.  Quantitative prediction of mental workload with the ACT-R cognitive architecture , 2012 .

[8]  C. Lebiere,et al.  An integrated theory of list memory. , 1998 .

[9]  Maya Cakmak,et al.  Optimality of human teachers for robot learners , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[10]  Susan R. Fussell,et al.  Comparing a computer agent with a humanoid robot , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[11]  Brian Scassellati,et al.  Discovering task constraints through observation and active learning , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Rainer Stiefelhagen,et al.  How to Transfer? Zero-Shot Object Recognition via Hierarchical Transfer of Semantic Attributes , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[13]  Bin Li,et al.  A survey on instance selection for active learning , 2012, Knowledge and Information Systems.

[14]  Christoph H. Lampert,et al.  Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Andrea Lockerd Thomaz,et al.  Robot Learning from Human Teachers , 2014, Robot Learning from Human Teachers.

[16]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[17]  Maya Cakmak,et al.  Power to the People: The Role of Humans in Interactive Machine Learning , 2014, AI Mag..

[18]  Jason Baldridge,et al.  How well does active learning actually work? Time-based evaluation of cost-reduction strategies for language documentation. , 2009, EMNLP.

[19]  Andrew McCallum,et al.  Reducing Labeling Effort for Structured Prediction Tasks , 2005, AAAI.

[20]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[21]  Brett Browning,et al.  Learning by demonstration with critique from a human teacher , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[22]  Peter Stone,et al.  Opportunistic Active Learning for Grounding Natural Language Descriptions , 2017, CoRL.

[23]  Anca D. Dragan,et al.  Active Preference-Based Learning of Reward Functions , 2017, Robotics: Science and Systems.

[24]  Daniel,et al.  Default Probability , 2004 .

[25]  Maya Cakmak,et al.  Transparent active learning for robots , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[26]  Samuel Kaski,et al.  Knowledge elicitation via sequential probabilistic inference for high-dimensional prediction , 2016, Machine Learning.

[27]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[28]  Maya Cakmak,et al.  Designing Interactions for Robot Active Learners , 2010, IEEE Transactions on Autonomous Mental Development.

[29]  David D. Lewis,et al.  Heterogeneous Uncertainty Sampling for Supervised Learning , 1994, ICML.

[30]  Christoph H. Lampert,et al.  Attribute-Based Classification for Zero-Shot Visual Object Categorization , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Bernt Schiele,et al.  Evaluating knowledge transfer and zero-shot learning in a large-scale setting , 2011, CVPR 2011.

[32]  Ville Kyrki,et al.  Active Robot Learning for Temporal Task Models , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[33]  Kristen Grauman,et al.  What's it going to cost you?: Predicting effort vs. informativeness for multi-label image annotations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  John R. Anderson.  How Can the Human Mind Occur in the Physical Universe , 2007 .

[35]  Stephanie Rosenthal,et al.  Acquiring Accurate Human Responses to Robots’ Questions , 2012, Int. J. Soc. Robotics.

[36]  David Cohn,et al.  Active Learning , 2010, Encyclopedia of Machine Learning.

[37]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[38]  Minoru Asada,et al.  Initiative in robot assistance during collaborative task execution , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[39]  Brian Scassellati,et al.  The Benefits of Interactions with Physically Present Robots over Video-Displayed Agents , 2011, Int. J. Soc. Robotics.

[40]  Andrea Lockerd Thomaz,et al.  Towards Intelligent Arbitration of Diverse Active Learning Queries , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[41]  Mukesh Singhal,et al.  Learning from Richer Human Guidance: Augmenting Comparison-Based Learning with Feature Queries , 2018, HRI.

[42]  Cynthia Breazeal,et al.  Training a Robot via Human Feedback: A Case Study , 2013, ICSR.