A Cognitive Robot Collaborative Reinforcement Learning Algorithm

A cognitive collaborative reinforcement learning algorithm (CCRL) that incorporates an advisor into the learning process is developed to improve supervised learning. An autonomous learner is enabled with a self awareness cognitive skill to decide when to solicit instructions from the advisor. The learner can also assess the value of advice, and accept or reject it. The method is evaluated for robotic motion planning using simulation. Tests are conducted for advisors with skill levels from expert to novice. The CCRL algorithm and a combined method integrating its logic with Clouse's Introspection Approach, outperformed a base-line fully autonomous learner, and demonstrated robust performance when dealing with various advisor skill levels, learning to accept advice received from an expert, while rejecting that of less skilled collaborators. Although the CCRL algorithm is based on RL, it fits other machine learning methods, since advisor's actions are only added to the outer layer. Keywords—Robot learning, human-robot collaboration, motion planning, reinforcement learning.

[1]  Taghi M. Khoshgoftaar,et al.  An application of a rule-based model in software quality classification , 2007, ICMLA 2007.

[2]  J. A. Clouse,et al.  An Introspection Approach to Querying a Trainer , 1996 .

[3]  Michael A. Goodrich,et al.  Experiments in adjustable autonomy , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[4]  E. Yael,et al.  Human-Robot Collaborative Learning System for Inspection , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[5]  Thomas G. Dietterich.  Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition , 1999, J. Artif. Intell. Res..

[6]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[7]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[8]  Victor Uc Cetina.  Supervised reinforcement learning using behavior models , 2007, Sixth International Conference on Machine Learning and Applications (ICMLA 2007).

[9]  Raymond J. Mooney,et al.  Using Active Relocation to Aid Reinforcement Learning , 2006, FLAIRS.

[10]  Manfred Huber,et al.  Learning from Reinforcement and Advice Using Composite Reward Functions , 2003, FLAIRS Conference.

[11]  Andrea Lockerd Thomaz,et al.  Learning from human teachers with Socially Guided Exploration , 2008, 2008 IEEE International Conference on Robotics and Automation.