Modeling agent behavior through online evolutionary and reinforcement learning

The process of creation and validation of an agent-based simulation model requires the modeler to undergo a number of prototyping, testing, analyzing and re-designing rounds. The aim is to specify and calibrate the proper low-level agent behavior that truly produces the intended macro-level phenomena. We assume that this development can be supported by agent learning techniques, specially by generating inspiration about behaviors as starting points for the modeler. In this contribution we address this learning-driven modeling task and compare two methods that are producing decision trees: reinforcement learning with a post-processing step for generalization and Genetic Programming.

[1]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[2]  Robert Junges,et al.  Learning convergence and agent behavior interpretation for designing agent-based simulations , 2010 .

[3]  Sotiris B. Kotsiantis,et al.  Supervised Machine Learning: A Review of Classification Techniques , 2007, Informatica.

[4]  Mal Lee,et al.  Learning enabled cooperative agent behavior in an evolutionary and competitive environment , 2006, Neural Computing & Applications.

[5]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[6]  Joshua M. Epstein,et al.  Growing Artificial Societies: Social Science from the Bottom Up , 1996 .

[7]  Gerhard Weiß,et al.  Adaptation and Learning in Multi-Agent Systems: Some Remarks and a Bibliography , 1995, Adaption and Learning in Multi-Agent Systems.

[8]  Sridhar Mahadevan,et al.  Automatic Programming of Behavior-Based Robots Using Reinforcement Learning , 1991, Artif. Intell..

[9]  Franziska Klügl-Frohnmeyer,et al.  Multiagent Simulation Model Design Strategies , 2009, MALLOW.

[10]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[11]  Jonathan Cagan,et al.  Evolutionary Multi-Agent Systems: An Adaptive and Dynamic Approach to Optimization , 2009 .

[12]  Tiago Francisco,et al.  Evolving predator and prey behaviours with co-evolution using genetic programming and decision trees , 2008, GECCO '08.

[13]  Jörg Denzinger,et al.  On Customizing Evolutionary Learning of Agent Behavior , 2004, Canadian Conference on AI.

[14]  Jörg Denzinger,et al.  Improving Evolutionary Learning of Cooperative Behavior by Including Accountability of Strategy Components , 2003, MATES.

[15]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[16]  Franziska Klügl-Frohnmeyer,et al.  Evaluation of Techniques for a Learning-Driven Modeling Methodology in Multiagent Simulation , 2010, MATES.

[17]  Jörg Denzinger,et al.  Improving modeling of other agents using tentative stereotypes and compactification of observations , 2004, Proceedings. IEEE/WIC/ACM International Conference on Intelligent Agent Technology, 2004. (IAT 2004)..

[18]  Steven M. Gustafson,et al.  Genetic Programming And Multi-agent Layered Learning By Reinforcements , 2002, GECCO.

[19]  Jörg Denzinger,et al.  Evolutionary online learning of cooperative behavior with situation-action pairs , 2000, Proceedings Fourth International Conference on MultiAgent Systems.

[20]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[21]  Franziska Klügl-Frohnmeyer,et al.  Evolution for modeling: a genetic programming framework for sesam , 2011, GECCO.