How do Modelers Code Artificial Societies? Investigating Practices and Quality of Netlogo Codes from Large Repositories

Many guidelines have been developed for simulations in general or for agent-based models which support artificial societies. When applying such guidelines to examine existing practices, assessment studies are limited by the artifacts released by modelers. Although code is the final product defining an artificial society, 90% of the code produced is not released hence previous assessments necessarily focused on higher-level items such as conceptual design or validation. We address this gap by collecting 338 artificial societies from two hosting platforms, CoMSES/OpenABM and GitHub. An innovation of our approach is the use of software engineering techniques to automatically examine the models with respect to items such as commenting the code, using libraries, or dividing code into functions. We found that developers of artificial societies code the decision-making of their agents from scratch in every model, despite the existence of several libraries that could be used as building blocks.

[1]  Philippe J. Giabbanelli,et al.  Solving Challenges at the Interface of Simulation and Big Data Using Machine Learning , 2019, 2019 Winter Simulation Conference (WSC).

[2]  Simon J. E. Taylor,et al.  APPLYING THE STRESS GUIDELINES FOR REPRODUCIBILITY IN MODELING & SIMULATION: APPLICATION TO A DISEASE MODELING CASE STUDY , 2018, 2018 Winter Simulation Conference (WSC).

[3]  Jacqueline L. Whalley,et al.  Measuring the difficulty of code comprehension tasks using software metrics , 2013, ACE '13.

[4]  Philippe J. Giabbanelli,et al.  Ideal, Best, and Emerging Practices in Creating Artificial Societies , 2019, Spring Simulation Multiconference.

[5]  Pierre Lemarinier,et al.  Agent Based Modelling and Simulation tools: A review of the state-of-art software , 2017, Comput. Sci. Rev..

[6]  David W. Binkley,et al.  Effective identifier names for comprehension and memory , 2007, Innovations in Systems and Software Engineering.

[7]  Gabriele Bavota,et al.  Automatically assessing code understandability: How far are we? , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[8]  F. Bianchi,et al.  Agent‐based models in sociology , 2015 .

[9]  Daniela E. Damian,et al.  The promises and perils of mining GitHub , 2009, MSR 2014.

[10]  I. Oren,et al.  Concepts and Criteria to Assess Acceptability of Simulation Studies : A Frame of Reference , 2001 .

[11]  Flaminio Squazzoni,et al.  Agent-Based Computational Sociology , 2012 .

[12]  Westley Weimer,et al.  Learning a Metric for Code Readability , 2010, IEEE Transactions on Software Engineering.

[13]  Bernard P. Zeigler,et al.  Theory of Modelling and Simulation , 1979, IEEE Transactions on Systems, Man and Cybernetics.

[14]  Philippe Mathieu,et al.  IODA: an interaction-oriented approach for multi-agent based simulations , 2010, Autonomous Agents and Multi-Agent Systems.

[15]  Gabriele Bavota,et al.  Automatically Assessing Code Understandability , 2019, IEEE Transactions on Software Engineering.

[16]  S. Railsback,et al.  The Evolution of Agent-based Simulation Platforms : A Review of NetLogo 5 . 0 and ReLogo , 2012 .

[17]  Charles M. Macal,et al.  Everything you need to know about agent-based modelling and simulation , 2016, J. Simulation.

[18]  Gregory R. Madey,et al.  Tools of the Trade: A Survey of Various Agent Based Modeling Platforms , 2009, J. Artif. Soc. Soc. Simul..

[19]  Marco A. Janssen,et al.  The Practice of Archiving Model Code of Agent-Based Models , 2017, J. Artif. Soc. Soc. Simul..

[20]  Samarth Swarup.  Adequacy: What Makes a Simulation Good Enough? , 2019, 2019 Spring Simulation Conference (SpringSim).

[21]  Bernard P. Zeigler,et al.  Theory of Modelling and Simulation , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[22]  Kathleen M. Carley.  Simulating Complex Social-Behavioral Systems , 2019, 2019 Spring Simulation Conference (SpringSim).

[23]  Premkumar T. Devanbu,et al.  A simpler model of software readability , 2011, MSR '11.

[24]  J. Barlow.  Problems and Promises , 2005 .

[25]  Ilias Sakellariou,et al.  Enhancing NetLogo to Simulate BDI Communicating Agents , 2008, SETN.

[26]  Philippe J. Giabbanelli,et al.  The Intersection of Agent Based Models and Fuzzy Cognitive Maps: A Review of an Emerging Hybrid Modeling Practice , 2019, 2019 Winter Simulation Conference (WSC).

[27]  Pietro Terna,et al.  How to build and use agent-based models in social science , 2000 .