Agent Learning Instead of Behavior Implementation for Simulations - A Case Study Using Classifier Systems

Although multi-agent simulations are an intuitive way of conceptualizing systems that consist of autonomous actors, a major problem is the actual design of the agent behavior. In this contribution, we examine the potential of using agent-based learning for implementing the agent behavior. We enhanced SeSAm, a platform for agent-based simulation, by replacing the usual rule-based agent architecture by XCS, a well-known learning classifier system (LCS). The resulting model is tested using a simple evacuation scenario. The results show that on the one hand side plausible agent behavior could be learned. On the other hand side, though, the results are quite brittle concerning the frame of environmental feedback, perception and action modeling.

[1]  Ester Bernadó-Mansilla,et al.  Bounding XCS's parameters for unbalanced datasets , 2006, GECCO '06.

[2]  Zahia Guessoum,et al.  Using adaptive multi-agent systems to simulate economic models , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[3]  Robert J. Collins,et al.  AntFarm: Towards Simulated Evolution , 2007 .

[4]  J. Grefenstette.  The Evolution of Strategies for Multi-agent Environments , 1987 .

[5]  Charles E. Taylor,et al.  Artificial Life II , 1991 .

[6]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[7]  John H. Holland,et al.  COGNITIVE SYSTEMS BASED ON ADAPTIVE ALGORITHMS1 , 1978 .

[8]  Martin V. Butz.  Combining Gradient-Based With Evolutionary Online Learning: An Introduction to Learning Classifier Systems , 2007, 7th International Conference on Hybrid Intelligent Systems (HIS 2007).

[9]  Pier Luca Lanzi,et al.  Learning classifier systems from a reinforcement learning perspective , 2002, Soft Comput..

[10]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[11]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[12]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[13]  Franziska Klügl-Frohnmeyer,et al.  Large-Scale Agent-Based Pedestrian Simulation , 2007, MATES.

[14]  James C. Spohrer,et al.  KidSim: programming agents without a programming language , 1994, CACM.

[15]  Donald A. Waterman,et al.  Pattern-Directed Inference Systems , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Maarten Peeters,et al.  Learning Automata as a Basis for Multi Agent Reinforcement Learning , 2005, EUMAS.

[17]  Sandip Sen,et al.  Learning and Adaption in Multi-Agent Systems , 2006 .

[18]  Sandip Sen,et al.  Adaption and Learning in Multi-Agent Systems , 1995, Lecture Notes in Computer Science.

[19]  David Stuart Robertson,et al.  Enacting the Distributed Business Workflows Using BPEL4WS on the Multi-agent Platform , 2005, MATES.

[20]  Franziska Klügl,et al.  Evolutionary Optimization of Societies in Simulated Multi-Agent Systems , 2000 .

[21]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[22]  Gerhard Weiß,et al.  Adaptation and Learning in Multi-Agent Systems: Some Remarks and a Bibliography , 1995, Adaption and Learning in Multi-Agent Systems.

[23]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[24]  Stewart W. Wilson.  Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[25]  Martin V. Butz,et al.  Rule-Based Evolutionary Online Learning Systems - A Principled Approach to LCS Analysis and Design , 2006, Studies in Fuzziness and Soft Computing.

[26]  Matthias Fuchs,et al.  Experiments in learning prototypical situations for variants of the pursuit game , 1999 .

[27]  C. Adami,et al.  Introduction To Artificial Life , 1997, IEEE Trans. Evol. Comput..

[28]  John J. Grefenstette,et al.  The Evolution of Strategies for Multiagent Environments , 1992, Adapt. Behav..

[29]  Richard S. Sutton,et al.  Dimensions of Reinforcement Learning , 1998 .

[30]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .