Alecsys and the AutonoMouse: Learning to control a real robot by distributed classifier systems

In this article we investigate the feasibility of using learning classifier systems as a tool for building adaptive control systems for real robots. Their use on real robots imposes efficiency constraints which are addressed by three main tools: parallelism, distributed architecture, and training. Parallelism is useful to speed up computation and to increase the flexibility of the learning system design. Distributed architecture helps in making it possible to decompose the overall task into a set of simpler learning tasks. Finally, training provides guidance to the system while learning, shortening the number of cycles required to learn. These tools and the issues they raise are first studied in simulation, and then the experience gained with simulations is used to implement the learning system on the real robot. Results have shown that with this approach it is possible to let the AutonoMouse, a small real robot, learn to approach a light source under a number of different noise and lesion conditions.

[1]  Michael J. Flynn,et al.  Some Computer Organizations and Their Effectiveness , 1972, IEEE Transactions on Computers.

[2]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[3]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[4]  George G. Robertson,et al.  Parallel Implementation of Genetic Algorithms in a Classifier Rystem , 1987, ICGA.

[5]  Rick L. Riolo,et al.  The Emergence of Default Hierarchies in Learning Classifier Systems , 1989, ICGA.

[6]  D.E. Goldberg,et al.  Classifier Systems and Genetic Algorithms , 1989, Artif. Intell..

[7]  Richard S. Sutton,et al.  Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming , 1990, ML.

[8]  Rodney A. Brooks,et al.  Elephants don't play chess , 1990, Robotics Auton. Syst..

[9]  R. A. Brooks,et al.  Intelligence without Representation , 1991, Artif. Intell..

[10]  Marco Dorigo,et al.  Alecsys: A Parallel Laboratory for Learning Classifier Systems , 1991, ICGA.

[11]  John J. Grefenstette,et al.  Lamarckian Learning in Multi-Agent Environments , 1991, ICGA.

[12]  Marco Dorigo,et al.  Using transputers to increase speed and flexibility of genetics-based machine learning systems , 1992, Microprocess. Microprogramming.

[13]  Rodney A. Brooks,et al.  Artificial Life and Real Robots , 1992 .

[14]  Sridhar Mahadevan,et al.  Automatic Programming of Behavior-Based Robots Using Reinforcement Learning , 1991, Artif. Intell..

[15]  L.-J. Lin,et al.  Hierarchical learning of robot skills by reinforcement , 1993, IEEE International Conference on Neural Networks.

[16]  Marco Dorigo,et al.  Genetics-based machine learning and behavior-based robotics: a new synthesis , 1993, IEEE Trans. Syst. Man Cybern..

[17]  Marco Dorigo,et al.  Genetic and Non-Genetic Operators in ALECSYS , 1993, Evolutionary Computation.

[18]  Long Ji Lin,et al.  Scaling Up Reinforcement Learning for Robot Control , 1993, International Conference on Machine Learning.

[19]  Marco Colombetti,et al.  The Role of the Trainer in Reinforcement Learning , 1994, COLT 1994.

[20]  Marco Colombetti,et al.  Robot Shaping: Developing Autonomous Agents Through Learning , 1994, Artif. Intell..

[21]  Marco Colombetti,et al.  Training Agents to Perform Sequential Behavior , 1994, Adapt. Behav..

[22]  Marco Dorigo,et al.  A comparison of Q-learning and classifier systems , 1994 .

[23]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..