Generating Inspiration for Multi-Agent Simulation Design by Q-Learning

One major challenge in developing multiagent simulations is to find the appropriate agent design that is able to generate the intended overall phenomenon dynamics, but does not contain unnecessary details. In this paper we suggest to use agent learning for supporting the development of an agent model: the modeler defines the environmental model and the agent interfaces. Using rewards capturing the intended agent behavior, reinforcement learning techniques can be used for learning the rules that are optimally governing the agent behavior. However, for really being useful in a modeling and simulation context, a human modeler must be able to review and understand the outcome of the learning. We propose to use additional forms of learning as post-processing step for supporting the analysis of the learned model. We test our ideas using a simple evacuation simulation scenario.

[1]  Yoichiro Maeda.  Simulation for behavior learning of multi-agent robot , 1998, J. Intell. Fuzzy Syst..

[2]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[3]  Franziska Klügl-Frohnmeyer,et al.  Agent-Based Pedestrian Simulation of Train Evacuation Integrating Environmental Data , 2009, KI.

[4]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[5]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[6]  Maarten Peeters,et al.  Learning Automata as a Basis for Multi Agent Reinforcement Learning , 2005, EUMAS.

[7]  Mal Lee,et al.  Learning enabled cooperative agent behavior in an evolutionary and competitive environment , 2006, Neural Computing & Applications.

[8]  Gerhard Weiß,et al.  Adaptation and Learning in Multi-Agent Systems: Some Remarks and a Bibliography , 1995, Adaption and Learning in Multi-Agent Systems.

[9]  Frank Puppe,et al.  Approaches for resolving the dilemma between model structure refinement and parameter calibration in agent-based simulations , 2006, AAMAS '06.

[10]  Gauthier Picard,et al.  Living design for open computational systems , 2003, WET ICE 2003. Proceedings. Twelfth IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises, 2003..

[11]  Franziska Klügl-Frohnmeyer,et al.  Multiagent Simulation Model Design Strategies , 2009, MALLOW.

[12]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[13]  Peter Clark,et al.  The CN2 induction algorithm , 2004, Machine Learning.

[14]  Martin V. Butz,et al.  Agent Learning Instead of Behavior Implementation for Simulations - A Case Study Using Classifier Systems , 2008, MATES.

[15]  Robert J. Collins,et al.  AntFarm: Towards Simulated Evolution , 2007 .

[16]  J. Grefenstette.  The Evolution of Strategies for Multi-agent Environments , 1987 .

[17]  Sridhar Mahadevan,et al.  Automatic Programming of Behavior-Based Robots Using Reinforcement Learning , 1991, Artif. Intell..

[18]  Matthias Fuchs,et al.  Experiments in learning prototypical situations for variants of the pursuit game , 1999 .

[19]  John J. Grefenstette,et al.  The Evolution of Strategies for Multiagent Environments , 1992, Adapt. Behav..

[20]  C. Adami,et al.  Introduction To Artificial Life , 1997, IEEE Trans. Evol. Comput..

[21]  R. Neruda,et al.  Performance Comparison of Relational Reinforcement Learning and RBF Neural Networks for Small Mobile Robots , 2008, 2008 Second International Conference on Future Generation Communication and Networking Symposia.