Ten iterative steps in development and evaluation of environmental models

Models are increasingly being relied upon to inform and support natural resource management. They are incorporating an ever broader range of disciplines and now often confront people without strong quantitative or model-building backgrounds. These trends imply a need for wider awareness of what constitutes good model-development practice, including reporting of models to users and sceptical review of models by users. To this end the paper outlines ten basic steps of good, disciplined model practice. The aim is to develop purposeful, credible models from data and prior knowledge, in consort with end-users, with every stage open to critical review and revision. Best practice entails identifying clearly the clients and objectives of the modelling exercise; documenting the nature (quantity, quality, limitations) of the data used to construct and test the model; providing a strong rationale for the choice of model family and features (encompassing review of alternative approaches); justifying the techniques used to calibrate the model; serious analysis, testing and discussion of model performance; and making a resultant statement of model assumptions, utility, accuracy, limitations, and scope for improvement. In natural resource management applications, these steps will be a learning process, even a partnership, between model developers, clients and other interested parties.

[1]  Anthony J. Jakeman,et al.  Participatory Modelling in Natural Resource Management: A Comparison of Four Case Studies , 2003 .

[2]  M. Crawley Non‐linear Regression , 2007 .

[3]  D. L. Scarnecchia,et al.  Fundamentals of Ecological Modelling , 1995 .

[4]  J. Wainwright,et al.  Environmental Modelling: Finding Simplicity in Complexity , 2013 .

[5]  Anthony J. Jakeman,et al.  Progress in integrated assessment and modelling , 2002, Environ. Model. Softw..

[6]  Hong Wang,et al.  Recursive estimation and time-series analysis , 1986, IEEE Trans. Acoust. Speech Signal Process..

[7]  Anthony J. Jakeman,et al.  Application of an adaptive method for integrated assessment of water allocation issues in the Namoi river catchment, Australia. , 2003 .

[8]  Carlo Giupponi,et al.  Sustainable management of water resources : an integrated approach , 2006 .

[9]  William K. Lauenroth,et al.  Models in Ecosystem Science , 2003, Models in Ecosystem Science.

[10]  P. E. Wellstead,et al.  Introduction to physical system modelling , 1979 .

[11]  M. B. Beck,et al.  Elucidating public perceptions of environmental behavior: a case study of Lake Lanier , 2005, Environ. Model. Softw..

[12]  Peter J. Gawthrop,et al.  Metamodelling: for bond graphs and dynamic systems , 1996 .

[13]  Denis Dochain,et al.  A simplified method to assess structurally identifiable parameters in Monod-based activated sludge models. , 2003, Water research.

[14]  Ayalew Kassahun,et al.  Quality assurance in model based water management - review of existing practice and outline of new approaches , 2005, Environ. Model. Softw..

[15]  S. Funtowicz,et al.  Combining Quantitative and Qualitative Measures of Uncertainty in Model‐Based Environmental Assessment: The NUSAP System , 2005, Risk analysis : an official publication of the Society for Risk Analysis.

[16]  S. Hawking A Brief History of Time: From the Big Bang to Black Holes , 1988 .

[17]  Daniel L. Schmoldt,et al.  Building Knowledge-Based Systems for Natural Resource Management , 1996, Springer US.

[18]  Warren E. Walker,et al.  Defining Uncertainty: A Conceptual Basis for Uncertainty Management in Model-Based Decision Support , 2003 .

[19]  Joanne E. Caminiti,et al.  Catchment modelling-a resource manager's perspective , 2004, Environ. Model. Softw..

[20]  Torsten Söderström,et al.  Model structure selection for multivariable systems by cross-validation methods , 1987 .

[21]  Richard Forsyth,et al.  Expert systems: Principles and case studies , 1984 .

[22]  W. C. Hession,et al.  Watershed-level uncertainties : Implications for phosphorus management and eutrophication , 2000 .

[23]  A. L. Kidd,et al.  Knowledge acquisition for expert systems: a practical handbook , 1987 .

[24]  Ralf Denzer,et al.  Generic integration of environmental decision support systems - state-of-the-art , 2005, Environ. Model. Softw..

[25]  Alison L. Kidd,et al.  Knowledge Acquisition: An Introductory Framework , 1987 .

[26]  R. Quentin Grafton,et al.  The economics of the environment and natural resources , 2004 .

[27]  Ari Jolma,et al.  Methods of uncertainty treatment in environmental models , 2005, Environ. Model. Softw..

[28]  Gill Begnor Concise encyclopaedia of environmental systems: Edited by Peter C. Young. Pergamon Press Ltd. 769 pp. ISBN: 0-08-036198-6 , 1994 .

[29]  Jerome R. Ravetz,et al.  Uncertainty and Quality in Science for Policy , 1990 .

[30]  Anand Patwardhan,et al.  Assessing integrated assessments , 1996 .

[31]  M. B. Beck,et al.  Modelling Change in Environmental Systems , 1995 .

[32]  A. Jakeman,et al.  How much complexity is warranted in a rainfall‐runoff model? , 1993 .

[33]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[34]  M. Bruce Beck Vulnerability of water quality in intensively developing urban watersheds , 2005, Environ. Model. Softw..

[35]  Stefano Marsili-Libelli,et al.  Identification of dynamic models for horizontal subsurface constructed wetlands , 2005 .

[36]  M.J.W. Jansen,et al.  Review of Saltelli, A. & Chan, K. & E.M.Scott (Eds) (2000), Sensitivity analysis. Wiley (2000) , 2001 .

[37]  Krist V. Gernaey,et al.  Activated sludge wastewater treatment plant modelling and simulation: state of the art , 2004, Environ. Model. Softw..

[38]  Arild Thowsen,et al.  Structural identifiability , 1977, 1977 IEEE Conference on Decision and Control including the 16th Symposium on Adaptive Processes and A Special Symposium on Fuzzy Set Theory and Applications.

[39]  J. Norton Optimal smoothing in the identification of linear time-varying systems , 1975 .

[40]  Ralf Seppelt Computer-Based Environmental Management , 2003 .

[41]  Sándor M. Veres,et al.  Structure selection of stochastic dynamic systems : the information criterion approach , 1991 .

[42]  Rik Pintelon,et al.  An Introduction to Identification , 2001 .

[43]  Peter Reichert,et al.  On the usefulness of overparameterized ecological models , 1997 .

[44]  N Oreskes,et al.  Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences , 1994, Science.

[45]  Eric Walter,et al.  Identification of Parametric Models: from Experimental Data , 1997 .

[46]  Richard Forsyth The expert systems phenomenon , 1989 .

[47]  John D. Bredehoeft,et al.  Ground-water models cannot be validated , 1992 .

[48]  K. J Versyck,et al.  Optimal experimental design for practical identification of unstructured growth models , 1997 .

[49]  Erik Mostert,et al.  Participation for Sustainable Water Management , 2006 .

[50]  S. Marsili-Libelli,et al.  Reliability of parameter estimation in respirometric models. , 2005, Water research.