PROGRAMMING AGENT BEHAVIOR BY LEARNING IN SIMULATION MODELS

Designing the proper agent behavior for a multiagent system is a complex task. Often it is not obvious which of the agents' actions, and the interactions among them and with their environment, can produce the intended macro-phenomenon. We assume that the modeler can benefit from using agent-learning techniques. There are several issues with which learning can help modeling; for example, by using self-adaptive agents for calibration. In this contribution we are dealing with another example: the usage of learning for supporting system analysis and model design. A candidate-learning architecture is the combination of reinforcement learning and decision tree learning. The former generates a policy for agent behavior and the latter is used for abstraction and interpretation purposes. Here, we focus on the relation between policy-learning convergence and the quality of the abstracted model produced from that.

[1]  Gauthier Picard,et al.  ADELFE: A Methodology for Adaptive Multi-agent Systems Engineering , 2002, ESAW.

[2]  Franziska Klügl-Frohnmeyer,et al.  Generating Inspiration for Multi-Agent Simulation Design by Q-Learning , 2010, MALLOW.

[3]  金田 重郎,et al.  C4.5: Programs for Machine Learning (書評) , 1995 .

[4]  Maarten Peeters,et al.  Learning Automata as a Basis for Multi Agent Reinforcement Learning , 2005, EUMAS.

[5]  Mal Lee,et al.  Learning enabled cooperative agent behavior in an evolutionary and competitive environment , 2006, Neural Computing & Applications.

[6]  Franziska Klügl-Frohnmeyer,et al.  Evaluation of Techniques for a Learning-Driven Modeling Methodology in Multiagent Simulation , 2010, MATES.

[7]  Gerhard Weiß,et al.  Adaptation and Learning in Multi-Agent Systems: Some Remarks and a Bibliography , 1995, Adaption and Learning in Multi-Agent Systems.

[8]  John J. Grefenstette,et al.  The Evolution of Strategies for Multiagent Environments , 1992, Adapt. Behav..

[9]  Jörg Denzinger,et al.  Evolutionary online learning of cooperative behavior with situation-action pairs , 2000, Proceedings Fourth International Conference on MultiAgent Systems.

[10]  Robert J. Collins,et al.  AntFarm: Towards Simulated Evolution , 2007 .

[11]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[12]  Christoph Endres,et al.  Introduction to Artificial Life , 2000, Künstliche Intell..

[13]  Forrest Stonedahl,et al.  Session details: Evolutionary computation and multi-agent systems and simulation (ECoMASS) , 2012, GECCO '12.

[14]  J. Grefenstette.  The Evolution of Strategies for Multi-agent Environments , 1987 .

[15]  Bart Baesens,et al.  Using Rule Extraction to Improve the Comprehensibility of Predictive Models , 2006 .

[16]  Jörg Denzinger,et al.  On Customizing Evolutionary Learning of Agent Behavior , 2004, Canadian Conference on AI.

[17]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[18]  Franziska Klügl-Frohnmeyer,et al.  Evolution for modeling: a genetic programming framework for sesam , 2011, GECCO.

[19]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[20]  Sridhar Mahadevan,et al.  Automatic Programming of Behavior-Based Robots Using Reinforcement Learning , 1991, Artif. Intell..

[21]  Larry D. Pyeatt,et al.  Decision Tree Function Approximation in Reinforcement Learning , 1999 .

[22]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[23]  R. Neruda,et al.  Performance Comparison of Relational Reinforcement Learning and RBF Neural Networks for Small Mobile Robots , 2008, 2008 Second International Conference on Future Generation Communication and Networking Symposia.

[24]  Andrea Lockerd Thomaz,et al.  Automatic State Abstraction from Demonstration , 2011, IJCAI.

[25]  Shimon Whiteson,et al.  Sample-Efficient Evolutionary Function Approximation for Reinforcement Learning , 2006, AAAI.

[26]  Franziska Klügl-Frohnmeyer,et al.  Modeling agent behavior through online evolutionary and reinforcement learning , 2011, 2011 Federated Conference on Computer Science and Information Systems (FedCSIS).