Measuring and Comparing Scalability of Agent-Based Simulation Frameworks

While computer simulation gained importance as a technique for generating knowledge in various research disciplines, the size of simulation models representing real world scenarios is growing, too. In Social Simulation, e.g., there is a need to simulate a large number of humans using individual software agents for generating and analyzing human-like behavior in artificial societies. Nowadays, a variety of toolkits and frameworks exists providing functionalities for supporting implementation and execution of simulation experiments. Yet, the choice of a suitable framework is difficult as unforeseen scalability issues may arise when extending agent models. Therefore, this paper aims at providing a method for analyzing and comparing agent-based simulation frameworks regarding their ability to scale simulation models and experiments. Based on performance metrics, standardized experiments are conducted while altering internal and external scaling parameters. As part of the study, four Java-based agent frameworks are analyzed and compared: Aimpulse Spectrum, JADE, MASON, and Repast.

[1]  Elyes Ben Hamida,et al.  Impact of the Physical Layer Modeling on the Accuracy and Scalability of Wireless Network Simulation , 2009, Simul..

[2]  Carl Hewitt,et al.  A Universal Modular ACTOR Formalism for Artificial Intelligence , 1973, IJCAI.

[3]  Victor R. Lesser,et al.  Farm: A Scalable Environment for Multi-agent Development and Evaluation , 2003, SELMAS.

[4]  Jan D. Gehrke,et al.  Towards Ontology-Based Multiagent Simulations: The Plasma Approach , 2010, ECMS.

[5]  Gianni A. Di Caro,et al.  Analysis of simulation environments for mobile ad hoc networks. , 2003 .

[6]  Henry H. Liu,et al.  Software Performance and Scalability - A Quantitative Approach , 2009, Wiley series on quantitative software engineering.

[7]  Mark D. Hill,et al.  What is scalability? , 1990, CARN.

[8]  Kalliopi Kravari,et al.  A Survey of Agent Platforms , 2015, J. Artif. Soc. Soc. Simul..

[9]  John B. Goodenough,et al.  On System Scalability , 2006 .

[10]  Ralph Deters.  Scalability & Multi-Agent Systems , 2001 .

[11]  Ingo J. Timm,et al.  Large scale multiagent simulation on the grid , 2005, CCGrid 2005. IEEE International Symposium on Cluster Computing and the Grid, 2005..

[12]  Steven L. Lytinen,et al.  Agent-based Simulation Platforms: Review and Development Recommendations , 2006, Simul..

[13]  Gn Gilbert,et al.  Simulating societies: an introduction , 1994 .

[14]  Aaron Helsinger,et al.  Cougaar: a scalable, distributed multi-agent architecture , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[15]  Jörg P. Müller,et al.  Application Impact of Multi-agent Systems and Technologies: A Survey , 2014, Agent-Oriented Software Engineering.

[16]  Robert Tobias,et al.  Evaluation of free Java-libraries for social-scientific agent based simulation , 2004, J. Artif. Soc. Soc. Simul..

[17]  Leon M. Tolbert,et al.  Scalable multi-agent system for real-time electric power management , 2001, 2001 Power Engineering Society Summer Meeting. Conference Proceedings (Cat. No.01CH37262).

[18]  L. C. Lee,et al.  The Stability, Scalability and Performance of Multi-agent Systems , 1998 .

[19]  Mariam Kiran,et al.  FLAME: simulating large populations of agents on parallel hardware architectures , 2010, AAMAS.

[20]  Ana Paula Appel,et al.  Large-Scale Multi-agent-Based Modeling and Simulation of Microblogging-Based Online Social Network , 2013, MABS.

[21]  Jaime Simão Sichman,et al.  MAS and Social Simulation: A Suitable Sommitment , 1998, MABS.

[22]  Paul Davidsson,et al.  Agent Based Social Simulation: A Computer Science View , 2002, J. Artif. Soc. Soc. Simul..

[23]  Nicholas R. Jennings,et al.  Intelligent agents: theory and practice , 1995, The Knowledge Engineering Review.

[24]  Maged M. Michael,et al.  Scale-up x Scale-out: A Case Study using Nutch/Lucene , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[25]  B. Edmonds,et al.  Social Simulation in the Social Sciences , 2014 .

[26]  Sandeep K. Singhal,et al.  Using projection aggregations to support scalability in distributed simulation , 1996, Proceedings of 16th International Conference on Distributed Computing Systems.

[27]  Steffen Straßburger,et al.  Scalability in distributed simulations of agent-based models , 2009, Proceedings of the 2009 Winter Simulation Conference (WSC).

[28]  Omer F. Rana,et al.  What is scalability in multi-agent systems? , 2000, AGENTS '00.

[29]  J. Banks,et al.  Handbook of Simulation , 1998 .

[30]  Rj Allan,et al.  Survey of Agent Based Modelling and Simulation Tools , 2009 .

[31]  Nicholas R. Jennings,et al.  Improving the Scalability of Multi-Agent Systems , 2000, Agents Workshop on Infrastructure for Multi-Agent Systems.