Learning Classifier System on a humanoid NAO robot in dynamic environments

We present a modified version of Extended Classifier System (XCS) on a humanoid NAO robot. The robot is capable of learning a complete, accurate, and maximally general map of an environment through evolutionary search and reinforcement learning. The standard alternation between explore and exploit trials is revised so that the robot relearns only when necessary. This modification makes the learning more effective and provides the XCS with external memory to evaluate the environmental change. Furthermore, it overcomes the drawbacks of learning rate settings in traditional XCS. A simple object seeking task is presented which demonstrates the desirable adaptivity of LCS for a sequential task on a real robot in dynamic environments.

[1]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[2]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[3]  Christopher Stone,et al.  For Real! XCS with Continuous-Valued Inputs , 2003, Evolutionary Computation.

[4]  Olgierd Unold,et al.  Self-adaptation of learning rate in XCS working in noisy and dynamic environments , 2011, Comput. Hum. Behav..

[5]  Larry Bull,et al.  A neural learning classifier system with self-adaptive constructivism , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[6]  Stewart W. Wilson.  Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[7]  Larry Bull,et al.  X-TCS: accuracy-based learning classifier system robotics , 2005, 2005 IEEE Congress on Evolutionary Computation.

[8]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[9]  Larry Bull,et al.  ZCS Redux , 2002, Evolutionary Computation.

[10]  Larry Bull,et al.  TCS Learning Classifier System Controller on a Real Robot , 2002, PPSN.

[11]  MSc PhD Tim Kovacs BA.  Strength or Accuracy: Credit Assignment in Learning Classifier Systems , 2004, Distinguished Dissertations.

[12]  Pier Luca Lanzi,et al.  An Analysis of Generalization in the XCS Classifier System , 1999, Evolutionary Computation.

[13]  Charles E. Taylor.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. Complex Adaptive Systems.John H. Holland , 1994 .

[14]  Stewart W. Wilson.  ZCS: A Zeroth Level Classifier System , 1994, Evolutionary Computation.

[15]  Fernando José Von Zuben,et al.  Analysing Learning Classifier Systems in Reactive and Non-reactive Robotic Tasks , 2007, IWLCS.

[16]  Petr Musílek,et al.  Enhanced learning classifier system for robot navigation , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Pier Luca Lanzi,et al.  Learning classifier systems: then and now , 2008, Evol. Intell..

[18]  Martin V. Butz,et al.  An algorithmic description of XCS , 2000, Soft Comput..

[19]  Seiji Yamada,et al.  Interactive classifier system for real robot learning , 2000, Proceedings 9th IEEE International Workshop on Robot and Human Interactive Communication. IEEE RO-MAN 2000 (Cat. No.00TH8499).

[20]  Yoshiki Uchikawa,et al.  A Fuzzy Classifier System for evolutionary learning of robot behaviors , 1998 .

[21]  Stewart W. Wilson.  Generalization in the XCS Classifier System , 1998 .

[22]  Martin V. Butz,et al.  How XCS evolves accurate classifiers , 2001 .

[23]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[24]  Daniele Loiacono,et al.  XCSLib : The XCS Classifier System Library , 2008 .