Towards better modelling and decision support: Documenting model development, testing, and analysis using TRACE

The potential of ecological models for supporting environmental decision making is increasingly acknowledged. However, it often remains unclear whether a model is realistic and reliable enough. Good practice for developing and testing ecological models has not yet been established. Therefore, TRACE, a general framework for documenting a model's rationale, design, and testing was recently suggested. Originally TRACE was aimed at documenting good modelling practice. However, the word ‘documentation’ does not convey TRACE's urgency. Therefore, we re-define TRACE as a tool for planning, performing, and documenting good modelling practice. TRACE documents should provide convincing evidence that a model was thoughtfully designed, correctly implemented, thoroughly tested, well understood, and appropriately used for its intended purpose. TRACE documents link the science underlying a model to its application, thereby also linking modellers and model users, for example stakeholders, decision makers, and developers of policies. We report on first experiences in producing TRACE documents. We found that the original idea underlying TRACE was valid, but to make its use more coherent and efficient, an update of its structure and more specific guidance for its use are needed. The updated TRACE format follows the recently developed framework of model ‘evaludation’: the entire process of establishing model quality and credibility throughout all stages of model development, analysis, and application. TRACE thus becomes a tool for planning, documenting, and assessing model evaludation, which includes understanding the rationale behind a model and its envisaged use. We introduce the new structure and revised terminology of TRACE and provide examples.

[1]  Steven F Railsback,et al.  Pattern-oriented modelling: a ‘multi-scope’ for predictive systems ecology , 2012, Philosophical Transactions of the Royal Society B: Biological Sciences.

[2]  Steven F. Railsback,et al.  ANALYSIS OF HABITAT‐SELECTION RULES USING ANINDIVIDUAL‐BASED MODEL , 2002 .

[3]  Steven F. Railsback,et al.  Agent-Based and Individual-Based Modeling: A Practical Introduction , 2011 .

[4]  P. Calow,et al.  Promises and problems for the new paradigm for risk assessment and an alternative approach involving predictive systems models , 2012, Environmental toxicology and chemistry.

[5]  Pernille Thorbek,et al.  REVIEW: Towards a systems approach for understanding honeybee decline: a stocktaking and synthesis of existing models , 2013, The Journal of applied ecology.

[6]  Volker Grimm,et al.  Mechanistic effect modeling for ecological risk assessment: Where to go from here? , 2013, Integrated environmental assessment and management.

[7]  Robert Luttik,et al.  Population level risk assessment: practical considerations for evaluation of population models from a risk assessor's perspective , 2012, Environmental Sciences Europe.

[8]  Andreas Focks,et al.  Integrating chemical fate and population-level effect models for pesticides at landscape scale: New options for risk assessment , 2014 .

[9]  V. Lakshmi Useless Arithmetic: Why Environmental Scientists Can't Predict the Future , 2007 .

[10]  Valery E. Forbes,et al.  Two pairs of eyes are better than one: Combining individual-based and matrix models for ecological risk assessment of chemicals , 2014 .

[11]  Uta Berger,et al.  Pattern-Oriented Modeling of Agent-Based Complex Systems: Lessons from Ecology , 2005, Science.

[12]  Franziska Kl A Validation Methodology for Agent-Based Simulations , 2008 .

[13]  Simone Wannemaker,et al.  The Double Helix A Personal Account Of The Discovery Of The Structure Of Dna , 2016 .

[14]  R. Sibly,et al.  An energy budget agent-based model of earthworm populations and its application to study the effects of pesticides , 2014, Ecological modelling.

[15]  R. Nisbet,et al.  Predicting Population Dynamics from the Properties of Individuals: A Cross-Level Test of Dynamic Energy Budget Theory , 2013, The American Naturalist.

[16]  J. Gareth Polhill,et al.  The ODD protocol: A review and first update , 2010, Ecological Modelling.

[17]  G. Huse Individual‐based Modeling and Ecology , 2008 .

[18]  Valery E. Forbes,et al.  Population-level consequences of spatially heterogeneous exposure to heavy metals in soil: An individual-based model of springtails , 2013 .

[19]  Saul I. Gass,et al.  Feature Article - Decision-Aiding Models: Validation, Assessment, and Related Issues for Policy Analysis , 1983, Oper. Res..

[20]  Birgit Müller,et al.  A standard protocol for describing individual-based and agent-based models , 2006 .

[21]  Roman Ashauer,et al.  CREAM: a European project on mechanistic effect models for ecological risk assessment of chemicals , 2009, Environmental science and pollution research international.

[22]  Franziska Klügl-Frohnmeyer,et al.  A validation methodology for agent-based simulations , 2008, SAC.

[23]  Howard M. Kanare,et al.  Writing the Laboratory Notebook , 1985 .

[24]  Volker Grimm,et al.  Ecological models supporting environmental decision making: a strategy for the future. , 2010, Trends in ecology & evolution.

[25]  Volker Grimm,et al.  Merging validation and evaluation of ecological models to ‘evaludation’: A review of terminology and a practical approach , 2014 .

[26]  Ulrich Dirnagl,et al.  Reprint: Good Laboratory Practice: Preventing Introduction of Bias at the Bench , 2009, Stroke.

[27]  Pernille Thorbek,et al.  Linking pesticide exposure and spatial dynamics: An individual-based model of wood mouse (Apodemus sylvaticus) populations in agricultural landscapes , 2013 .