Replacing Rules by Neural Networks A Framework for Agent-Based Modelling

Agent-based modelling is a successful technique in many different fields of science. As a bottom-up method, it is able to simulate complex behaviour based on simple rules and show results at both micro and macro scales. However, developing agent-based models is not always straightforward. The most difficult step is defining the rules for the agent behaviour, since one often has to rely on many simplifications and assumptions in order to describe the complicated decision making processes. In this paper, we investigate the idea of building a framework for agent-based modelling that relies on an artificial neural network to depict the decision process of the agents. As a proof of principle, we use this framework to reproduce Schelling’s segregation model. We show that it is possible to use the presented framework to derive an agent-based model without the need of manually defining rules for agent behaviour. Beyond reproducing Schelling’s model, we show expansions that are possible due to the framework, such as training the agents in a different environment, which leads to different agent behaviour.

[1]  Andrew M. Colman,et al.  The complexity of cooperation: Agent-based models of competition and collaboration , 1998, Complex..

[2]  Khashayar Khorasani,et al.  Multi-agent team cooperation: A game theory approach , 2009, Autom..

[3]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[4]  Sebastian Thrun,et al.  Efficient Exploration In Reinforcement Learning , 1992 .

[5]  D. Foley,et al.  The economy needs agent-based modelling , 2009, Nature.

[6]  Christian Hofer,et al.  Large scale simulation of CO 2 emissions caused by urban car traffic: An agent-based network approach , 2018 .

[7]  Bruce Edmonds,et al.  Sociology and Social Theory in Agent Based Social Simulation: A Symposium , 2001, Comput. Math. Organ. Theory.

[8]  Fred Collopy,et al.  How effective are neural networks at forecasting and prediction? A review and evaluation , 1998 .

[9]  M. Macy,et al.  FROM FACTORS TO ACTORS: Computational Sociology and Agent-Based Modeling , 2002 .

[10]  Jörg P. Müller,et al.  A Cooperation Model for Autonomous Agents , 1996, ATAL.

[11]  T. Schelling.  Models of Segregation , 1969 .

[12]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[13]  Jürgen Schmidhuber,et al.  Training Very Deep Networks , 2015, NIPS.

[14]  Jean-Pierre Nadal,et al.  Phase diagram of a Schelling segregation model , 2009, 0903.4694.

[15]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[16]  Nicholas R. Jennings,et al.  Agent-Based Computing: Promise and Perils , 1999, IJCAI.

[17]  R. Leombruni,et al.  Why are economists sceptical about agent-based simulations? , 2005 .

[18]  Le Zhang,et al.  A survey of randomized algorithms for training neural networks , 2016, Inf. Sci..

[19]  Flaminio Squazzoni,et al.  Agent-Based Computational Sociology , 2012 .

[20]  Hans M. Amman,et al.  Handbook of Computational Economics , 1996 .

[21]  Yoshua Bengio,et al.  Training deep neural networks with low precision multiplications , 2014 .

[22]  Herbert Dawid,et al.  EURACE: A massively parallel agent-based model of the European economy , 2008, Appl. Math. Comput..

[23]  Stefan Fritsch,et al.  neuralnet: Training of Neural Networks , 2010, R J..

[24]  Siti Zaiton Mohd Hashim,et al.  Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm , 2012, Appl. Math. Comput..

[25]  S. Parsons,et al.  Game Theory and Decision Theory in Agent-Based Systems , 2002, Multiagent Systems, Artificial Societies, and Simulated Organizations.

[26]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[27]  Thomas C. Schelling,et al.  Dynamic models of segregation , 1971 .

[28]  Geoffrey E. Hinton,et al.  Training Recurrent Neural Networks , 2013 .

[29]  F. Benjamin Zhan,et al.  Agent-based modelling and simulation of urban evacuation: relative effectiveness of simultaneous and staged evacuation strategies , 2008, J. Oper. Res. Soc..

[30]  Leigh Tesfatsion,et al.  Agent-Based Computational Economics: Growing Economies From the Bottom Up , 2002, Artificial Life.

[31]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[32]  Lynne Hamill,et al.  Social Circles: A Simple Structure for Agent-Based Social Network Models , 2009, J. Artif. Soc. Soc. Simul..

[33]  Bo Chen,et al.  A Review of the Applications of Agent Technology in Traffic and Transportation Systems , 2010, IEEE Transactions on Intelligent Transportation Systems.

[34]  O P Judson,et al.  The rise of the individual-based model in ecology. , 1994, Trends in ecology & evolution.

[35]  Pietro Terna,et al.  How to build and use agent-based models in social science , 2000 .

[36]  Kai Nagel,et al.  Towards truly agent-based traffic and mobility simulations , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[37]  Lucila Ohno-Machado,et al.  Logistic regression and artificial neural network classification models: a methodology review , 2002, J. Biomed. Informatics.

[38]  Risto Miikkulainen,et al.  Efficient Reinforcement Learning Through Evolving Neural Network Topologies , 2002, GECCO.

[39]  Andrea Bonarini,et al.  Reinforcement Learning in Continuous Action Spaces through Sequential Monte Carlo Methods , 2007, NIPS.

[40]  Paul Davidsson,et al.  Agent Based Social Simulation: A Computer Science View , 2002, J. Artif. Soc. Soc. Simul..

[41]  Euel Elliott,et al.  Exploring cooperation and competition using agent-based modeling , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[42]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[43]  Bruce Edmonds,et al.  Errors and Artefacts in Agent-Based Modelling , 2009, J. Artif. Soc. Soc. Simul..

[44]  Lukas Furst.  Cities And Complexity Understanding Cities With Cellular Automata Agent Based Models And Fractals , 2016 .

[45]  Kevin T. Comer,et al.  An agent-based model of network effects on tax compliance and evasion , 2014 .

[46]  Narasimhan Sundararajan,et al.  A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation , 2005, IEEE Transactions on Neural Networks.

[47]  Dirk Helbing,et al.  Agent-Based Modeling , 2012 .

[48]  Eric Bonabeau,et al.  Agent-based modeling: Methods and techniques for simulating human systems , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[49]  William A. V. Clark,et al.  Understanding the social context of the Schelling segregation model , 2008, Proceedings of the National Academy of Sciences.

[50]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[51]  Arend Hintze,et al.  Evolutionary game theory using agent-based methods. , 2014, Physics of life reviews.

[52]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.

[53]  Joshua M. Epstein,et al.  Agent-based computational models and generative social science , 1999, Complex..

[54]  F. Bianchi,et al.  Agent‐based models in sociology , 2015 .

[55]  Sebastian Thrun,et al.  Monte Carlo POMDPs , 1999, NIPS.

[56]  L. Tesfatsion.  Agent-based computational economics : A constructive approach to economic theory , 2006 .

[57]  Steven D. Brown,et al.  Data processing using neural networks , 1993 .