Socially Guided XCS: Using Teaching Signals to Boost Learning

In this paper, we show how we can improve task learning by using social interaction to guide the learning process of a robot, in a Human-Robot Interaction scenario. We introduce a novel method that simultaneously learns a social reward function on the teaching signals provided by a human and uses it to bootstrap task learning. We propose a model we call the Socially Guided XCS, based on the XCS framework, and we evaluate it in simulation with respect to the standard XCS algorithm. We show that our model improves the learning speed of XCS.

[1]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[2]  Eduardo F. Morales,et al.  Dynamic Reward Shaping: Training a Robot by Voice , 2010, IBERAMIA.

[3]  Sonia Chernova,et al.  Effect of human guidance and state space size on Interactive Reinforcement Learning , 2011, 2011 RO-MAN.

[4]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[5]  Peter Stone,et al.  A social reinforcement learning agent , 2001, AGENTS '01.

[6]  Pierre-Yves Oudeyer,et al.  Interactive Learning from Unlabeled Instructions , 2014, UAI.

[7]  Stewart W. Wilson,et al.  Noname manuscript No. (will be inserted by the editor) Learning Classifier Systems: A Survey , 2022 .

[8]  Daniele Loiacono,et al.  XCSLib : The XCS Classifier System Library , 2008 .

[9]  Andrea Lockerd Thomaz,et al.  Reinforcement Learning with Human Teachers: Evidence of Feedback and Guidance with Implications for Learning Performance , 2006, AAAI.

[10]  Terrence J Sejnowski,et al.  Foundations for a New Science of Learning , 2009, Science.

[11]  Andrea Lockerd Thomaz,et al.  Policy Shaping: Integrating Human Feedback with Reinforcement Learning , 2013, NIPS.

[12]  Farbod Fahimi,et al.  Online human training of a myoelectric prosthesis controller via actor-critic reinforcement learning , 2011, 2011 IEEE International Conference on Rehabilitation Robotics.

[13]  Masaki Ogino,et al.  Cognitive Developmental Robotics: A Survey , 2009, IEEE Transactions on Autonomous Mental Development.

[14]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[15]  Peter Stone,et al.  Combining manual feedback with subsequent MDP reward signals for reinforcement learning , 2010, AAMAS.

[16]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..