RatKit: Repeatable Automated Testing Toolkit for Agent-Based Modeling and Simulation

Agent-based modeling and simulation (ABMS) became an attractive and efficient way to model large-scale complex systems. The use of models always raises the question whether the model is correctly encoded (verification) and accurately represents the real system (validation). However, achieving a sufficiently credible agent-based simulation (ABS) model is still difficult due to weak verification, validation and testing (VV&T) techniques. Moreover, there is no comprehensive and integrated toolkit for VV&T of ABS models that demonstrates that inaccuracies exist and/or which reveals the existing errors in the model. Based on this observation, we designed and developed RatKit: a toolkit for ABS models to conduct VV&T. RatKit facilitates the VV&T process of ABMS by providing an integrated environment that allows repeatable and automated execution of ABS tests. This paper presents RatKit in detail and demonstrates its effectiveness by showing its applicability on a simple well-known case study: predator - prey.

[1]  Luis Antunes,et al.  Multi-Agent-Based Simulation VII, International Workshop, MABS 2006, Hakodate, Japan, May 8, 2006, Revised and Invited Papers , 2007, MABS.

[2]  Benoit Gaudou,et al.  GAMA: A Spatially Explicit, Multi-level, Agent-Based Modeling and Simulation Platform , 2013, PAAMS.

[3]  Muaz A. Niazi,et al.  A Novel Agent-Based Simulation Framework for Sensing in Complex Adaptive Environments , 2011, IEEE Sensors Journal.

[4]  Phil McMinn,et al.  Towards the Automatic Identification of Faulty Multi-Agent Based Simulation Runs Using MASTER , 2012, MABS.

[5]  Kent L. Beck,et al.  Test-driven Development - by example , 2002, The Addison-Wesley signature series.

[6]  Ilene Burnstein,et al.  Practical Software Testing , 2003, Springer Professional Computing.

[7]  Jack P. C. Kleijnen,et al.  EUROPEAN JOURNAL OF OPERATIONAL , 1992 .

[8]  Osman Balci.  Golden Rules of Verification, Validation, Testing, and Certification of Modeling and Simulation Applications , 2010 .

[9]  Muaz A. Niazi,et al.  Agent-based computing from multi-agent systems to agent-based models: a visual survey , 2011, Scientometrics.

[10]  Takao Terano.  Exploring the Vast Parameter Space of Multi-Agent Based Simulation , 2006, MABS.

[11]  Michael Lees,et al.  Validation of Agent-Based Simulation through Human Computation: An Example of Crowd Simulation , 2011, MABS.

[12]  Osman Balci.  Principles and techniques of simulation validation, verification, and testing , 1995, WSC '95.

[13]  Yves Demazeau,et al.  Advances on Practical Applications of Agents and Multi-Agent Systems - 10th International Conference on Practical Applications of Agents and Multi-Agent Systems, PAAMS 2012, Salamanca, Spain, 28-30 March, 2012 , 2012, Advances in Intelligent and Soft Computing.

[14]  Ben H. Thacker,et al.  Concepts of Model Verification and Validation , 2004 .

[15]  Steven F. Railsback,et al.  Agent-Based and Individual-Based Modeling: A Practical Introduction , 2011 .

[16]  Jaime Simão Sichman,et al.  Multi-Agent-Based Simulation VI , 2005, Lecture Notes in Computer Science.

[17]  Oguz Dikenelli,et al.  Mimicking human neuronal pathways in silico: an emergent model on the effective connectivity , 2013, Journal of Computational Neuroscience.

[18]  Garold Stasser,et al.  Computer simulation as a research tool: The DISCUSS model of group decision making , 1988 .

[19]  Franziska Kl.  A Validation Methodology for Agent-Based Simulations , 2008 .

[20]  Salima Hassas,et al.  Self-Organisation: Paradigms and Applications , 2003, Engineering Self-Organising Systems.

[21]  Franziska Klügl-Frohnmeyer,et al.  A validation methodology for agent-based simulations , 2008, SAC.

[22]  Joshua M. Epstein,et al.  Agent-based computational models and generative social science , 1999, Complex..

[23]  Jordi Sabater-Mir,et al.  Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics): Preface , 2012 .

[24]  Guillaume Hutzler,et al.  Automatic Tuning of Agent-Based Models Using Genetic Algorithms , 2005, MABS.

[25]  Gildas Morvan,et al.  A Methodology to Engineer and Validate Dynamic Multi-level Multi-agent Based Simulations , 2012, MABS.

[26]  Oguz Dikenelli,et al.  A generic testing framework for agent-based simulation models , 2011, 2011 Federated Conference on Computer Science and Information Systems (FedCSIS).

[27]  Francesca Giardini,et al.  Multi-Agent-Based Simulation XIII , 2012, Lecture Notes in Computer Science.

[28]  Tom De Wolf,et al.  Emergence Versus Self-Organisation: Different Concepts but Promising When Combined , 2004, Engineering Self-Organising Systems.

[29]  Uta Berger,et al.  Pattern-Oriented Modeling of Agent-Based Complex Systems: Lessons from Ecology , 2005, Science.

[30]  Osman Balci.  Validation, verification, and testing techniques throughout the life cycle of a simulation study , 1994, WSC '94.