Uncertainty Assessment in Agent-Based Simulation: An Exploratory Study

This paper presents an overview of uncertainty assessment in agent-based simulations, mainly related to land use and cover change. Almost every multiagent-based simulation review has expressed the need for statistical methods to evaluate the certainty of the results. Yet these problems continue to be underestimated and often neglected. This work aims to review how uncertainty is being portrayed in agent-based simulation and to perform an exploratory study to use statistical methods to estimate uncertainty. MASE, a Multi-Agent System for Environmental simulation, is the system under study. We first identified the most sensitive parameters using Morris One-at-a-Time sensitivity analysis. The efforts to assess agent-based simulation through statistical methods are paramount to corroborate and improve the level of confidence of the research that has been made in land use simulation.

[1]  J. D. Morrison,et al.  Evaluating prediction uncertainty in simulation models , 1999 .

[2]  Roland W. Scholz,et al.  Feedback loops and types of adaptation in the modelling of land-use decisions in an agent-based simulation , 2012, Environ. Model. Softw..

[3]  Daniel G. Brown,et al.  Empirical characterisation of agent behaviours in socio-ecological systems , 2011, Environ. Model. Softw..

[4]  Raymond R. Hill,et al.  A Survey of Agent-Based Modeling Practices (January 1998 to July 2008) , 2009, J. Artif. Soc. Soc. Simul..

[5]  Eric Koomen,et al.  Comparing the input, output, and validation maps for several models of land change , 2008 .

[6]  Célia Ghedini Ralha,et al.  MASE-BDI: agent-based simulator for environmental land change with efficient and parallel auto-tuning , 2016, Applied Intelligence.

[7]  Robert E. Marks,et al.  Validating Simulation Models: A General Framework and Four Applied Examples , 2007 .

[8]  D. Steinberg,et al.  Computer experiments: a review , 2010 .

[9]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[10]  Célia Ghedini Ralha,et al.  Mase: A Multi-Agent-Based Environmental Simulator , 2017 .

[11]  John L. Casti,et al.  Complexification: Explaining a Paradoxical World Through the Science of Surprise , 1994 .

[12]  Matthias Meyer,et al.  Opening the ‘black box’ of simulations: increased transparency and effective communication through the systematic design of experiments , 2011, Computational and Mathematical Organization Theory.

[13]  D. Goldsman,et al.  Output analysis procedures for computer simulations , 2000, 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165).

[14]  Anthony J. Jakeman,et al.  Selecting among five common modelling approaches for integrated environmental assessment and management , 2013, Environ. Model. Softw..

[15]  Wei Chu,et al.  A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrological model , 2014, Environ. Model. Softw..

[16]  Francis W. Zwiers,et al.  Guidance Note for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties , 2010 .

[17]  J. Gareth Polhill,et al.  The ODD protocol: A review and first update , 2010, Ecological Modelling.

[18]  Michael Luck,et al.  MC2MABS: A Monte Carlo Model Checker for Multiagent-Based Simulations , 2015, MABS.

[19]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .

[20]  Doina Olaru,et al.  Fuzzy Logic for Social Simulation Using NetLogo , 2015, J. Artif. Soc. Soc. Simul..

[21]  B. Iooss,et al.  A Review on Global Sensitivity Analysis Methods , 2014, 1404.2405.

[22]  Célia Ghedini Ralha,et al.  A multi-agent model system for land-use change simulation , 2013, Environ. Model. Softw..

[23]  Forrest Stonedahl,et al.  The Complexities of Agent-Based Modeling Output Analysis , 2015, J. Artif. Soc. Soc. Simul..

[24]  Roger David Braddock,et al.  The use of graph theory in the sensitivity analysis of the model output: a second order screening method , 1999 .

[25]  Jack P. C. Kleijnen,et al.  State-of-the-Art Review: A User's Guide to the Brave New World of Designing Simulation Experiments , 2005, INFORMS J. Comput..

[26]  Wei Gong,et al.  Assessing parameter importance of the Common Land Model based on qualitative and quantitative sensitivity analysis , 2013 .

[27]  J. Mas,et al.  Benchmarking of LUCC modelling tools by various validation techniques and error analysis , 2014 .

[28]  Birgit Müller,et al.  A standard protocol for describing individual-based and agent-based models , 2006 .

[29]  Subramanian Ramamoorthy,et al.  Are You Doing What I Think You Are Doing? Criticising Uncertain Agent Models , 2015, UAI.