Using Classifier Systems as Adaptive Expert Systems for Control

In complex simulations involving several interacting agents, the behavior of the overall program is diffcult to predict and control. As a consequence, the designers have to adopt a trial-and-error strategy. In this paper we want to show that helping experts to design simulation automata as classifier systems (CSs) by hand and using a semi-automated improvement functionality can be a very effcient engineering approach. Through the example of a simple multiagent simulation, we show how simulation automata can be implemented into the CS formalism. Then we explain how the obtained CS can be improved either by hand or thanks to adaptive algorithms. We first show how giving indications on the non-Markov character of the problems faced by the classifiers can help the experts to improve the controllers and we explain why adding modularity in the CS formalism is important. Then we show how the adaptive algorithms inherent to Learning Classifier Systems (LCSs) can be used in such a context, we discuss our methodology and we present an experimental study of the effciency of this approach. Finally, we point to diffculties raised by our perspective, we present directions for future research and conclude.

[1]  Stephen Cameron,et al.  Robot sheepdog project achieves automatic flock control , 1998 .

[2]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[3]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[4]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[5]  Andrew McCallum,et al.  Overcoming Incomplete Perception with Utile Distinction Memory , 1993, ICML.

[6]  Olivier Sigaud,et al.  YACS: Combining Dynamic Programming with Generalization in Classifier Systems , 2000, IWLCS.

[7]  Stewart W. Wilson.  Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[8]  Jürgen Schmidhuber,et al.  HQ-Learning , 1997, Adapt. Behav..

[9]  Rick L. Riolo,et al.  Lookahead planning and latent learning in a classifier system , 1991 .

[10]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[11]  R. Bellman.  Dynamic programming. , 1957, Science.

[12]  Long Ji Lin,et al.  Reinforcement Learning of Non-Markov Decision Processes , 1995, Artif. Intell..

[13]  Pier Luca Lanzi.  Adding memory to XCS , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[14]  Marco Dorigo,et al.  Genetic and Non-Genetic Operators in ALECSYS , 1993, Evolutionary Computation.

[15]  Olivier Sigaud,et al.  Being Reactive by Exchanging Roles: An Empirical Study , 2000, Balancing Reactivity and Social Deliberation in Multi-Agent Systems.

[16]  Stewart W. Wilson.  ZCS: A Zeroth Level Classifier System , 1994, Evolutionary Computation.

[17]  Randall D. Beer,et al.  Evolving Dynamical Neural Networks for Adaptive Behavior , 1992, Adapt. Behav..

[18]  Jean-Yves Donnart.  Architecture cognitive et proprietes adaptatives d'un animat motivationnellement autonome , 1998 .