Can we replicate real human behaviour using artificial neural networks?

Agent-based modelling is a powerful tool when simulating human systems, yet when human behaviour cannot be described by simple rules or maximising one’s own profit, we quickly reach the limits of this methodology. Machine learning has the potential to bridge this gap by providing a link between what people observe and how they act in order to reach their goal. In this paper we use a framework for agent-based modelling that utilizes human values like fairness, conformity and altruism. Using this framework we simulate a public goods game and compare to experimental results. We can report good agreement between simulation and experiment and furthermore find that the presented framework outperforms strict reinforcement learning. Both the framework and the utility function are generic enough that they can be used for arbitrary systems, which makes this method a promising candidate for a foundation of a universal agent-based model.

[1]  John A. Sokolowski,et al.  Applying reinforcement learning to an insurgency Agent-based Simulation , 2014 .

[2]  Atsuo Kato,et al.  Deep Reinforcement Learning in Agent Based Financial Market Simulation , 2020, Journal of Risk and Financial Management.

[3]  Gary E. Bolton,et al.  Chapter 59 Self-centered Fairness in Games with More Than Two Players , 2008 .

[4]  Linda Steg,et al.  Psychological research and global climate change , 2015 .

[5]  Christophe Le Page,et al.  Using agent-based modelling to simulate social-ecological systems across scales , 2019, GeoInformatica.

[6]  Pratap Chandra Sen,et al.  Supervised Classification Algorithms in Machine Learning: A Survey and Review , 2019, Advances in Intelligent Systems and Computing.

[7]  Fionn Murtagh,et al.  Multilayer perceptrons for classification and regression , 1991, Neurocomputing.

[8]  Giorgio Fagiolo,et al.  Validation of Agent-Based Models in Economics and Finance , 2019, Simulation Foundations, Methods and Applications.

[9]  Mamta Juneja,et al.  Machine Learning: A Review of the Algorithms and Its Applications , 2019 .

[10]  Abbas Ahmadi,et al.  Simulation-based optimization of radiotherapy: Agent-based modeling and reinforcement learning , 2017, Math. Comput. Simul..

[11]  K. N. Laland,et al.  The Biological Bases of Conformity , 2012, Front. Neurosci..

[12]  Petrônio C. L. Silva,et al.  COVID-ABS: An agent-based model of COVID-19 epidemic to simulate health and economic effects of social distancing interventions , 2020, Chaos, Solitons & Fractals.

[13]  Thomas Dietz,et al.  Altruism, self-interest, and energy consumption , 2015, Proceedings of the National Academy of Sciences.

[14]  R. Dorfman.  A Formula for the Gini Coefficient , 1979 .

[15]  J. Piliavin,et al.  Altruism: A Review of Recent Theory and Research , 1990 .

[16]  Branko Milanovic,et al.  A simple way to calculate the gini coefficient, and some implications , 1997 .

[17]  Klaus M. Schmidt,et al.  Fairness, Incentives, and Contractual Choices , 1999 .

[18]  C. Betsch.  How behavioural science data helps mitigate the COVID-19 crisis , 2020, Nature Human Behaviour.

[19]  Amir Abbas Rassafi,et al.  A hierarchical agent-based approach to simulate a dynamic decision-making process of evacuees using reinforcement learning , 2021 .

[20]  H. Gintis.  Beyond Homo economicus: evidence from experimental economics , 2000 .

[21]  Sara S. Metcalf,et al.  Linking models of human behaviour and climate alters projected climate change , 2018, Nature Climate Change.

[22]  Stewart T. Chang,et al.  Covasim: An agent-based model of COVID-19 dynamics and interventions , 2020, medRxiv.

[23]  Pierre Lemarinier,et al.  Agent Based Modelling and Simulation tools: A review of the state-of-art software , 2017, Comput. Sci. Rev..

[24]  Richard H. Thaler.  Journal of Economic Perspectives—Volume 14, Number 1—Winter 2000—Pages 133–141 From Homo Economicus to Homo Sapiens , 2022 .

[25]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[26]  Jan Lorenz,et al.  The triple‐filter bubble: Using agent‐based modelling to test a meta‐theoretical framework for the emergence of filter bubbles and echo chambers , 2018, The British journal of social psychology.

[27]  William T. Harbaugh,et al.  The Carrot or the Stick: Rewards, Punishments and Cooperation , 2002 .

[28]  F. Warneken,et al.  The developmental foundations of human fairness , 2017, Nature Human Behaviour.

[29]  Igor Nikolic,et al.  Agent-Based Modelling of Socio-Technical Systems , 2012, Agent-Based Social Systems.

[30]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[31]  U. Fischbacher,et al.  Are People Conditionally Cooperative? Evidence from a Public Goods Experiment , 2001 .

[32]  Giuseppe Bruno,et al.  Applications of agent-based models for optimization problems: A literature review , 2012, Expert Syst. Appl..

[33]  Julie C. Coultas,et al.  When in Rome... An Evolutionary Perspective on Conformity , 2004 .

[34]  Georg Jäger,et al.  Using Neural Networks for a Universal Framework for Agent-based Models , 2021, Mathematical and Computer Modelling of Dynamical Systems.

[35]  Suzana Dragicevic,et al.  Simulation and validation of a reinforcement learning agent-based model for multi-stakeholder forest management , 2010, Comput. Environ. Urban Syst..

[36]  Thomas C. Schelling,et al.  Dynamic models of segregation , 1971 .

[37]  Charles M. Macal,et al.  Everything you need to know about agent-based modelling and simulation , 2016, J. Simulation.

[38]  L. An,et al.  Modeling human decisions in coupled human and natural systems : Review of agent-based models , 2012 .

[39]  Hazel R. Parry,et al.  Large Scale Agent-Based Modelling: A Review and Guidelines for Model Scaling , 2012 .

[40]  Garrett Milam,et al.  Public goods provision by asymmetric agents: experimental evidence , 2013, Soc. Choice Welf..

[41]  Toon Haer,et al.  Advancing disaster policies by integrating dynamic adaptive behaviour in risk assessments using an agent-based modelling approach , 2019, Environmental Research Letters.

[42]  Ingook Jang,et al.  An Agent-Based Simulation Modeling with Deep Reinforcement Learning for Smart Traffic Signal Control , 2018, 2018 International Conference on Information and Communication Technology Convergence (ICTC).

[43]  Georg Jäger,et al.  Replacing Rules by Neural Networks A Framework for Agent-Based Modelling , 2019, Big Data Cogn. Comput..

[44]  Yevgeniy Vorobeychik,et al.  Empirically grounded agent-based models of innovation diffusion: a critical review , 2016, Artificial Intelligence Review.

[45]  F. Willekens,et al.  Decision-Making in Agent-Based Models of Migration: State of the Art and Challenges , 2016, European Journal of Population.

[46]  Joshua M. Epstein,et al.  Growing Artificial Societies: Social Science from the Bottom Up , 1996 .

[47]  David Demeritt,et al.  Climate Change and Society , 2012 .

[48]  Emmanuel Barillot,et al.  PhysiBoSS: a multi-scale agent-based modelling framework integrating physical dimension and cell signalling , 2018, bioRxiv.

[49]  Reza Ebrahimpour,et al.  Decentralized multi-agent based energy management of microgrid using reinforcement learning , 2020 .

[50]  David G. Rand,et al.  Risking Your Life without a Second Thought: Intuitive Decision-Making and Extreme Altruism , 2014, PloS one.

[51]  Ernst Fehr,et al.  Human altruism: economic, neural, and evolutionary perspectives , 2004, Current Opinion in Neurobiology.

[52]  Pietro Terna,et al.  How to build and use agent-based models in social science , 2000 .

[53]  David G. Rand,et al.  Using social and behavioural science to support COVID-19 pandemic response , 2020, Nature Human Behaviour.

[54]  Oleksandr Makeyev,et al.  Neural network with ensembles , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[55]  E. Fehr.  A Theory of Fairness, Competition and Cooperation , 1998 .

[56]  W. Hamilton,et al.  The evolution of cooperation. , 1984, Science.

[57]  Shane D. Johnson,et al.  State of the Art in Agent-Based Modeling of Urban Crime: An Overview , 2018, Journal of Quantitative Criminology.

[58]  S. O'Neill,et al.  Engaging the public with climate change: Behaviour change and communication , 2010 .

[59]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[60]  U. Fischbacher,et al.  The nature of human altruism , 2003, Nature.

[61]  Erik Cuevas,et al.  An agent-based model to evaluate the COVID-19 transmission risks in facilities , 2020, Computers in Biology and Medicine.

[62]  Nick Chater,et al.  Facing up to the uncertainties of COVID-19 , 2020, Nature Human Behaviour.

[63]  P. Hansen,et al.  Agent-based modelling and socio-technical energy transitions: A systematic literature review , 2019, Energy Research & Social Science.

[64]  Yaneer Bar-Yam,et al.  Segregation dynamics with reinforcement learning and agent based modeling , 2019, Scientific Reports.