Model testing and assessment: Perspectives from a swarm intelligence, agent-based model of forest insect infestations

Abstract Model testing procedures represent a major challenge in the development of agent-based models (ABMs). However, they are required stages for a model to be accepted and to serve as a forecasting, management or decision-making tool. This study presents a comprehensive approach for testing ForestSimMPB, an agent-based model (ABM) designed to simulate mountain pine beetle (MPB), Dendroctonus ponderosae Hopkins, outbreaks at the scale of individual trees. ForestSimMPB is a complex system model that is using swarming intelligence, capable to represent individuals’ behaviours and spatial interactions that influence their surrounding environment. Swarm Intelligence (SI) methods are integrated into the ABM in order to reproduce the collective reasoning and indirect communication of autonomous agents representing MPB behaviour within the forest environment. Model testing approach consist of verification, calibration, sensitivity analysis, validation and qualification stages. Model testing is accomplished by simulating MPB infestations using both the ForestSimMPB model and a Random–ABM model that serves as a null model. Outcomes comparison and assessment are performed using raster-based techniques as well as spatial metrics. Aerial photographs of the British Columbia, Canada study sites are used in this model testing approach. Results indicate that ForestSimMPB model representations of MPB outbreaks are more similar than Random model representations to the spatial distribution of MPB-dead trees.

[1]  A. Carroll,et al.  The biology and epidemiology of the mountain pine beetle in lodgepole pine forests. , 2006 .

[2]  N. Gotelli.  Null model analysis of species co-occurrence patterns , 2000 .

[3]  Alex Hagen-Zanker,et al.  An improved Fuzzy Kappa statistic that accounts for spatial autocorrelation , 2009, Int. J. Geogr. Inf. Sci..

[4]  Steven M. Manson,et al.  Challenges in Evaluating Models of Geographic Complexity , 2007 .

[5]  Alex Hagen-Zanker,et al.  Neutral models of landscape change as benchmarks in the assessment of model performance , 2008 .

[6]  Michael Batty,et al.  Ucl Centre for Advanced Spatial Analysis Working Papers Series Key Challenges in Agent-based Modelling for Geo-spatial Simulation Paper 121 -sept 07 Key Challenges in Agent-based Modelling for Geo-spatial Simulation , 2022 .

[7]  Steven M. Manson,et al.  Case studies, cross-site comparisons, and the challenge of generalization: comparing agent-based models of land-use change in frontier regions , 2008, Journal of land use science.

[8]  Roger White,et al.  Pattern based map comparisons , 2006, J. Geogr. Syst..

[9]  N Oreskes,et al.  Evaluation (not validation) of quantitative models. , 1998, Environmental health perspectives.

[10]  E. Johnson,et al.  Plant Disturbance Ecology: The Process and the Response , 2007 .

[11]  Jim Hardie,et al.  Pheromones of Non-Lepidopteran Insects Associated with Agricultural Plants , 1999 .

[12]  Andrew G. Birt,et al.  Simulating the impacts of southern pine beetle and fire on the dynamics of xerophytic pine landscapes in the southern Appalachians , 2007 .

[13]  Andrew Fall,et al.  A domain-specific language for models of landscape dynamics , 2001 .

[14]  Hans Visser,et al.  The Map Comparison Kit , 2006, Environ. Model. Softw..

[15]  Giles M. Foody,et al.  Status of land cover classification accuracy assessment , 2002 .

[16]  Manfred J. Lexer,et al.  Modelling bark beetle disturbances in a large scale forest scenario model to assess climate change impacts and evaluate adaptive management strategies , 2009 .

[17]  Michael J de Smith,et al.  Geospatial Analysis: A Comprehensive Guide to Principles, Techniques and Software Tools , 2007 .

[18]  Edward J. Rykiel,et al.  Testing ecological models: the meaning of validation , 1996 .

[19]  Piotr Jankowski,et al.  Agent-Based Models as Laboratories for Spatially Explicit Planning Policies , 2007 .

[20]  Roger White,et al.  Validating and Calibrating Integrated Cellular Automata Based Models of Land Use Change , 2008 .

[21]  Suzana Dragicevic,et al.  Assessing cellular automata model behaviour using a sensitivity analysis approach , 2006, Comput. Environ. Urban Syst..

[22]  Suzana Dragicevic,et al.  Simulation and validation of a reinforcement learning agent-based model for multi-stakeholder forest management , 2010, Comput. Environ. Urban Syst..

[23]  Lars Håkanson,et al.  A model to calculate heavy metal load to lakes dominated by urban runoff and diffuse inflow , 2001 .

[24]  Hong Li,et al.  Revealing spatial pattern dynamics in aquatic ecosystem modelling with Multi-Agent Systems in Lake Veluwe , 2010, Ecol. Informatics.

[25]  Xiaoping Liu,et al.  Embedding sustainable development strategies in agent‐based models for use as a planning tool , 2008, Int. J. Geogr. Inf. Sci..

[26]  James Croft,et al.  Victoria, British Columbia , 2011 .

[27]  Peter van Oosterom,et al.  Computers, Environment and Urban Systems , 2009 .

[28]  William Rand,et al.  Exurbia from the bottom-up: Confronting empirical challenges to characterizing a complex system , 2008 .

[29]  Roger White,et al.  Hierarchical fuzzy pattern matching for the regional comparison of land use maps , 2001, Int. J. Geogr. Inf. Sci..

[30]  Jorge J. Gómez-Sanz,et al.  Validation and Verification of Multi-agent Systems , 2002 .

[31]  Gary R. Graves,et al.  Macroecological signals of species interactions in the Danish avifauna , 2010, Proceedings of the National Academy of Sciences.

[32]  R. Pontius,et al.  Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment , 2011 .

[33]  H. Van Dyke Parunak,et al.  Swarming methods for geospatial reasoning , 2006, Int. J. Geogr. Inf. Sci..

[34]  Lasse Møller-Jensen,et al.  Agent‐based modelling of shifting cultivation field patterns, Vietnam , 2006, Int. J. Geogr. Inf. Sci..

[35]  Laura Painton Swiler,et al.  Calibration, validation, and sensitivity analysis: What's what , 2006, Reliab. Eng. Syst. Saf..

[36]  Suzana Dragicevic,et al.  Integrating high resolution remote sensing, GIS and fuzzy set theory for identifying susceptibility areas of forest insect infestations , 2005 .

[37]  K. McGarigal,et al.  FRAGSTATS: spatial pattern analysis program for quantifying landscape structure. , 1995 .

[38]  Arika Ligmann-Zielinska,et al.  Applying time-dependent variance-based global sensitivity analysis to represent the dynamics of an agent-based model of land use change , 2010, Int. J. Geogr. Inf. Sci..

[39]  Danielle J. Marceau,et al.  How humans shape wolf behavior in Banff and Kootenay National Parks, Canada , 2010 .

[40]  Stephen J. Walsh,et al.  An agent-based model of household dynamics and land use change , 2008 .

[41]  Barry Boots,et al.  Categorical maps, comparisons, and confidence , 2006, J. Geogr. Syst..

[42]  Lael Parrott,et al.  Conceptualization and implementation of a multi-agent model to simulate whale-watching tours in the St. Lawrence Estuary in Quebec, Canada , 2007, Environ. Model. Softw..

[43]  Suzana Dragicevic,et al.  A fuzzy-constrained cellular automata model of forest insect infestations , 2006 .

[44]  Toke T. Høye,et al.  Opening the black box—Development, testing and documentation of a mechanistically rich agent-based model , 2010 .

[45]  Jennifer L. Dungan,et al.  Focusing on feature-based differences in map comparison , 2006, J. Geogr. Syst..

[46]  Suzana Dragicevic,et al.  Agent-Based Model Validation Using Bayesian Networks and Vector Spatial Data , 2009 .

[47]  Justin Heavilin,et al.  16 – Dynamics of Mountain Pine Beetle Outbreaks , 2007 .

[48]  Paul W. Box,et al.  An individual-based model of canid populations: modelling territoriality and social structure , 2003 .

[49]  Alison J. Heppenstall,et al.  Crime reduction through simulation: An agent-based model of burglary , 2010, Comput. Environ. Urban Syst..

[50]  R. Pontius,et al.  Accuracy Assessment for a Simulation Model of Amazonian Deforestation , 2007 .

[51]  Michael Batty,et al.  Cities and Complexity: Understanding Cities Through Cellular Automata, Agent-Based Models and Fractals , 2005 .

[52]  Jens Christian Refsgaard,et al.  Modelling guidelinesterminology and guiding principles , 2004 .

[53]  Sean R Connolly,et al.  Process‐Based Models of Species Distributions and the Mid‐Domain Effect , 2005, The American Naturalist.

[54]  Klaus von Gadow,et al.  Sustainable Forest Management , 2000, Managing Forest Ecosystems.

[55]  Hao Chen,et al.  Diagnostic tools to evaluate a spatial land change projection along a gradient of an explanatory variable , 2010, Landscape Ecology.

[56]  Suzana Dragicevic,et al.  Modeling mountain pine beetle infestation with an agent-based approach at two spatial scales , 2010, Environ. Model. Softw..

[57]  Uldis Silins,et al.  Growth and crown efficiency of height repressed lodgepole pine; are suppressed trees more efficient? , 2004, Trees.

[58]  Uta Berger,et al.  Pattern-Oriented Modeling of Agent-Based Complex Systems: Lessons from Ecology , 2005, Science.

[59]  D. DeAngelis,et al.  Individual-based modeling of ecological and evolutionary processes , 2005 .

[60]  H. Van Dyke Parunak,et al.  Pheromone learning for self-organizing agents , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[61]  D. Geiszler,et al.  Modeling the dynamics of mountain pine beetle aggregation in a lodgepole pine stand , 2004, Oecologia.

[62]  Kenneth F. Raffa,et al.  Mixed messages across multiple trophic levels: the ecology of bark beetle chemical communication systems , 2001, CHEMOECOLOGY.

[63]  Elizabeth A. Fulton,et al.  An agent-based modelling approach to evaluation of multiple-use management strategies for coastal marine ecosystems , 2008, Math. Comput. Simul..

[64]  Craig A. Aumann,et al.  A methodology for developing simulation models of complex systems , 2007 .

[65]  Suzana Dragicevic,et al.  ForestSimMPB: A swarming intelligence and agent-based modeling approach for mountain pine beetle outbreaks , 2011, Ecol. Informatics.

[66]  Jesse A. Logan,et al.  Localized spatial and temporal attack dynamics of the mountain pine beetle in lodgepole pine. Forest Service research paper , 1996 .

[67]  B. Wilson,et al.  The mountain pine beetle: a synthesis of biology, management and impacts on lodgepole pine. , 2006 .

[68]  A Hagen-Zanker,et al.  Measuring the performance of geosimulation models by map comparison , 2008 .

[69]  Itzhak Benenson,et al.  The Dilemma of On-Street Parking Policy: Exploring Cruising for Parking Using an Agent-Based Model , 2010 .

[70]  N Oreskes,et al.  Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences , 1994, Science.

[71]  Steven F. Railsback,et al.  ANALYSIS OF HABITAT‐SELECTION RULES USING ANINDIVIDUAL‐BASED MODEL , 2002 .

[72]  William Rand,et al.  Path dependence and the validation of agent‐based spatial models of land use , 2005, Int. J. Geogr. Inf. Sci..

[73]  Michael Batty,et al.  Cities and complexity - understanding cities with cellular automata, agent-based models, and fractals , 2007 .

[74]  Michael Batty,et al.  Modelling and prediction in a complex world , 2005 .

[75]  Andries Petrus Engelbrecht,et al.  Fundamentals of Computational Swarm Intelligence , 2005 .