Multi-agent modeling and simulation in the AI age

With the rapid development of artificial intelligence (AI) technology and its successful application in various fields, modeling and simulation technology, especially multi-agent modeling and simulation (MAMS), of complex systems has rapidly advanced. In this study, we first describe the concept, technical advantages, research steps, and research status of MAMS. Then we review the development status of the hybrid modeling and simulation combining multi-agent and system dynamics, the modeling and simulation of multi-agent reinforcement learning, and the modeling and simulation of large-scale multi-agent. Lastly, we introduce existing MAMS platforms and their comparative studies. This work summarizes the current research situation of MAMS, thus helping scholars understand the systematic technology development of MAMS in the AI era. It also paves the way for further research on MAMS technology.

[1]  Weihua Gui,et al.  Visual perception-based fault diagnosis in froth flotation using statistical approaches , 2021, Tsinghua Science and Technology.

[2]  Manuela M. Veloso,et al.  Multiagent Systems: A Survey from a Machine Learning Perspective , 2000, Auton. Robots.

[3]  Zhao Zhi.  Reinforcement Learning Technology in Multi-Agent System , 2004 .

[4]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[5]  Kay W. Axhausen,et al.  An Agent-Based Microsimulation Model of Swiss Travel: First Results , 2003 .

[6]  Shimon Whiteson,et al.  Counterfactual Multi-Agent Policy Gradients , 2017, AAAI.

[7]  Farnad Nasirzadeh,et al.  System dynamics approach for construction risk analysis , 2008 .

[8]  Zhikun Ding,et al.  System Dynamics versus Agent-Based Modeling: A Review of Complexity Simulation in Construction Waste Management , 2018, Sustainability.

[9]  Giuseppe Bruno,et al.  Applications of agent-based models for optimization problems: A literature review , 2012, Expert Syst. Appl..

[10]  R. Palmer,et al.  Asset Pricing Under Endogenous Expectations in an Artificial Stock Market , 1996 .

[11]  Hazel R. Parry,et al.  Large Scale Agent-Based Modelling: A Review and Guidelines for Model Scaling , 2012 .

[12]  W. Arthur,et al.  The Economy as an Evolving Complex System II , 1988 .

[13]  Franziska Klügl-Frohnmeyer,et al.  Large-Scale Agent-Based Pedestrian Simulation , 2007, MATES.

[14]  Hongbo Li,et al.  Consensus of Second-Order Multi-Agent Systems with Time-Varying Delays and Antagonistic Interactions , 2015 .

[15]  Andrew Adamatzky,et al.  Artificial Life Models in Software , 2005 .

[16]  Howard M. Schwartz,et al.  The residual gradient FACL algorithm for differential games , 2015, 2015 IEEE 28th Canadian Conference on Electrical and Computer Engineering (CCECE).

[17]  Agostinho C. Rosa,et al.  Towards a standard model for research in agent-based modeling and simulation , 2015, PeerJ Prepr..

[18]  Warren Dixon,et al.  Optimal Adaptive Control and Differential Games by Reinforcement Leanring Principles [Book review] , 2014, IEEE Control Systems.

[19]  Matthew E. Taylor,et al.  A survey and critique of multiagent deep reinforcement learning , 2019, Autonomous Agents and Multi-Agent Systems.

[20]  Wenguang Chen,et al.  Heterogeneous parallel algorithm design and performance optimization for WENO on the Sunway Taihulight supercomputer , 2020 .

[21]  Stephen P. Ellner,et al.  Embedding System Dynamics in Agent Based Models for Complex Adaptive Systems , 2011, IJCAI.

[22]  Alexander Verbraeck,et al.  Including public transportation into a large-scale agent-based model for epidemic prediction and control , 2015, SummerSim.

[23]  Patrick Giraudoux,et al.  Coupling agent-based with equation-based models to study spatially explicit megapopulation dynamics , 2018, Ecological Modelling.

[24]  Dai Jin-hai.  Summarization of Agent-Based Modeling and Simulation , 2008 .

[25]  Michael J. North,et al.  AGENT-BASED MODELING AND SIMULATION: DESKTOP ABMS , 2007 .

[26]  Kamil Khadiev,et al.  On quantum methods for machine learning problems part I: Quantum tools , 2020, Big Data Min. Anal..

[27]  Charles M. Macal,et al.  To agent-based simulation from System Dynamics , 2010, Proceedings of the 2010 Winter Simulation Conference.

[28]  Bo Zhao,et al.  ZenLDA: Large-scale topic model training on distributed data-parallel platform , 2018, Big Data Min. Anal..

[29]  Karl Tuyls,et al.  Evolutionary Dynamics of Multi-Agent Learning: A Survey , 2015, J. Artif. Intell. Res..

[30]  Hazel R. Parry,et al.  A comparative analysis of parallel processing and super-individual methods for improving the computational performance of a large individual-based model , 2008 .

[31]  Agostinho C. Rosa,et al.  Parallelization Strategies for Spatial Agent-Based Models , 2015, International Journal of Parallel Programming.

[32]  Tobias Lorenz,et al.  Towards an orientation framework in multi-paradigm modeling Aligning purpose, object and methodology in System Dynamics, Agent-based Modeling and Discrete-Event-Simulation , 2006 .

[33]  Hazhir Rahmandad,et al.  Heterogeneity and Network Structure in the Dynamics of Diffusion: Comparing Agent-Based and Differential Equation Models , 2004, Manag. Sci..

[34]  J. Casti.  Would-Be Worlds: How Simulation Is Changing the Frontiers of Science , 1996 .

[35]  Luis M. Camarinha-Matos,et al.  A System Dynamics and Agent-Based Approach to Model Emotions in Collaborative Networks , 2017, DoCEIS.

[36]  Toyotaro Suzumura,et al.  Towards billion-scale social simulations , 2014, Proceedings of the Winter Simulation Conference 2014.

[37]  James J. Nutaro,et al.  Emulating a System Dynamics Model with Agent-Based Models: A Methodological Case Study in Simulation of Diabetes Progression , 2015 .

[38]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[39]  Filippos Christianos,et al.  Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning , 2019, ArXiv.

[40]  Salima Hassas,et al.  A Massive Multi-agent System for Brain MRI Segmentation , 2004, MMAS.

[41]  Ndaona Chokani,et al.  Large-scale multi-agent mobility simulations on a GPU: towards high performance and scalability , 2019, ANT/EDI40.

[42]  Amlan Mukherjee,et al.  Using Agent-Based Modeling to Study Construction Labor Productivity as an Emergent Property of Individual and Crew Interactions , 2009 .

[43]  R. G. Ingalls,et al.  Agent-Based Modeling and Simulation , 2017, Encyclopedia of Machine Learning and Data Mining.

[44]  Wu Xiao-ping.  Realization of ACS in Repast , 2007 .

[45]  Codrin Pasca,et al.  A comparison between system dynamics and agent based modeling and opportunities for cross-fertilization , 2007, 2007 Winter Simulation Conference.

[46]  Toyoaki Nishida,et al.  Massive Multi-agent systems control , 2004, FAABS.

[47]  Worawan Marurngsith,et al.  Creating GPU-Enabled Agent-Based Simulations Using a PDES Tool , 2013, DCAI.

[48]  Tobias Lorenz,et al.  Abductive Fallacies with Agent-Based Modeling and System Dynamics , 2009, EPOS.

[49]  Joshua M. Epstein,et al.  Growing Artificial Societies: Social Science from the Bottom Up , 1996 .

[50]  Leslie Pack Kaelbling,et al.  Effective reinforcement learning for mobile robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[51]  Han-Xiong Li,et al.  Data-based Suboptimal Neuro-control Design with Reinforcement Learning for Dissipative Spatially Distributed Processes , 2014 .

[52]  Michael J. North,et al.  Parallel agent-based simulation with Repast for High Performance Computing , 2013, Simul..

[53]  D. Wu,et al.  Modeling technological innovation risks of an entrepreneurial team using system dynamics: An agent-based perspective , 2010 .

[54]  Clinton Heinze,et al.  Thinking Quickly: Agents for Modeling Air Warfare , 1998, Australian Joint Conference on Artificial Intelligence.

[55]  Sarit Kraus,et al.  Making friends on the fly: Cooperating with new teammates , 2017, Artif. Intell..

[56]  Wenchen Yang,et al.  Large-Scale Agent-Based Transport Simulation in Shanghai, China , 2013 .

[57]  Simaan M. AbouRizk,et al.  Modeling Framework and Architecture of Hybrid System Dynamics and Discrete Event Simulation for Construction , 2011, Comput. Aided Civ. Infrastructure Eng..

[58]  Saeid Nahavandi,et al.  Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications , 2018, IEEE Transactions on Cybernetics.

[59]  Zongqing Lu,et al.  Learning Attentional Communication for Multi-Agent Cooperation , 2018, NeurIPS.

[60]  Hans Jochen Scholl,et al.  Agent-based and system dynamics modeling: a call for cross study and joint research , 2001, Proceedings of the 34th Annual Hawaii International Conference on System Sciences.

[61]  Euel Elliott,et al.  Adaptive agents, intelligence, and emergent human organization: Capturing complexity through agent-based modeling , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[62]  John Pourdehnad,et al.  System Dynamics and Intelligent Agent-Based Simulation : Where is the Synergy ? , 2022 .

[63]  Dorian Kodelja,et al.  Multiagent cooperation and competition with deep reinforcement learning , 2015, PloS one.

[64]  Jacob Andreas,et al.  Can Deep Reinforcement Learning Solve Erdos-Selfridge-Spencer Games? , 2017, ICML.

[65]  Ahmed Khalafallah,et al.  Neural Network Based Model for Predicting Housing Market Performance , 2008 .

[66]  W. Marsden.  I and J , 2012 .

[67]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[68]  Gabriel de Oliveira Ramos,et al.  Experience Sharing Between Cooperative Reinforcement Learning Agents , 2019, ICTAI.

[69]  David Silver,et al.  A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning , 2017, NIPS.

[70]  Reinhard German,et al.  Hybrid simulation with loosely coupled system dynamics and agent-based models for Prospective Health Technology Assessments , 2012, Proceedings Title: Proceedings of the 2012 Winter Simulation Conference (WSC).

[71]  Andreas Größler,et al.  Emergent structures in supply chains - a study integrating agent-based and system dynamics modeling , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[72]  Pierrick Tranouez,et al.  A multiagent urban traffic simulation , 2010, ArXiv.

[73]  Steven E. Phelan,et al.  A Note on the Correspondence Between Complexity and Systems Theory , 1999 .

[74]  Rob Fergus,et al.  Learning Multiagent Communication with Backpropagation , 2016, NIPS.

[75]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[76]  Yu Zhang,et al.  Intelligent cognitive spectrum collaboration: Convergence of spectrum sensing, spectrum access, and coding technology , 2020 .

[77]  Wojciech M. Czarnecki,et al.  Grandmaster level in StarCraft II using multi-agent reinforcement learning , 2019, Nature.

[78]  Peter Milling,et al.  Modeling the Forest or Modeling the Trees - A Comparison of System Dynamics and Agent-Based Simulation , 2003 .

[79]  Peter Stone,et al.  Autonomous agents modelling other agents: A comprehensive survey and open problems , 2017, Artif. Intell..

[80]  N. Basu,et al.  Growing a market economy , 1997 .

[81]  Per Hilletofth,et al.  Hybrid simulation models - When, Why, How? , 2010, Expert Syst. Appl..

[82]  Hlynur Stefansson,et al.  Integrated Agent-based and System Dynamics Modelling for Simulation of Sustainable Mobility , 2013 .

[83]  J Swanson,et al.  Business Dynamics—Systems Thinking and Modeling for a Complex World , 2002, J. Oper. Res. Soc..

[84]  N. Basu,et al.  Development of Aspen: A microanalytic simulation model of the US economy , 1996 .

[85]  R. Roy.  System dynamics , 2019, Information Systems.

[86]  Milind Tambe,et al.  Soar-RWA: Planning, Teamwork, and Intelligent Behavior for Synthetic Rotary Wing Aircraft , 1998 .

[87]  Farnad Nasirzadeh,et al.  A hybrid simulation framework for modelling construction projects using agent-based modelling and system dynamics: an application to model construction workers' safety behavior , 2018 .

[88]  Rob Fergus,et al.  Modeling Others using Oneself in Multi-Agent Reinforcement Learning , 2018, ICML.

[89]  Jay W. Forrester,et al.  System dynamics, systems thinking, and soft OR , 1994 .

[90]  Uwe Aickelin,et al.  Juxtaposition of System Dynamics and Agent-Based Simulation for a Case Study in Immunosenescence , 2015, PloS one.

[91]  Kristina Lerman,et al.  Massive Multi-agent Data-Driven Simulations of the GitHub Ecosystem , 2019, PAAMS.

[92]  Ken R. McNaught,et al.  Design classes for hybrid simulations involving agent-based and system dynamics models , 2012, Simul. Model. Pract. Theory.

[93]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[94]  Nadine Schieritz,et al.  Integrating System Dynamics and Agent-Based Modeling , 2002 .

[95]  Yi Wu,et al.  Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments , 2017, NIPS.

[96]  Hao Xu,et al.  Decentralized Adaptive Optimal Control for Massive Multi-agent Systems Using Mean Field Game with Self-Organizing Neural Networks , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[97]  Yoav Shoham,et al.  If multi-agent learning is the answer, what is the question? , 2007, Artif. Intell..

[98]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[99]  Mohammad Hashem Haghighat,et al.  Intrusion detection system using voting-based neural network , 2021, Tsinghua Science and Technology.