Practical identifiability analysis of environmental models.

Identifiability of a system model can be considered as the extent to which one can capture its parameter values from observational data and other prior knowledge of the system. Identifiability must be considered in context so that the objectives of the modelling must also be taken into account in its interpretation. A model may be identifiable for certain objective functions but not others; its identifiability may depend not just on the model structure but also on the level and type of noise, and may even not be identifiable when there is no noise on the observational data. Context also means that non-identifiability might not matter in some contexts, such as when representing pluralistic values among stakeholders, and may be very important in others, such as where it leads to intolerable uncertainties in model predictions. Uncertainty quantification of environmental systems is receiving increasing attention especially through the development of sophisticated methods, often statistically-based. This is partly driven by the desire of society and its decision makers to make more informed judgments as to how systems are better managed and associated resources efficiently allocated. Less attention seems to be given by modellers to understand the imperfections in their models and their implications. Practical methods of identifiability analysis can assist greatly here to assess if there is an identifiability problem so that one can proceed to decide if it matters, and if so how to go about modifying the model (transforming parameters, selecting specific data periods, changing model structure, using a more sophisticated objective function). A suite of relevant methods is available and the major useful ones are discussed here including sensitivity analysis, response surface methods, model emulation and the quantification of uncertainty. The paper also addresses various perspectives and concepts that warrant further development and use.

[1]  J. P. Norton,et al.  Normal-mode identifiability analysis of linear compartmental systems in linear stages , 1980 .

[2]  M J Chappell,et al.  Extensions to a procedure for generating locally identifiable reparameterisations of unidentifiable systems. , 2000, Mathematical biosciences.

[3]  Clifford H. Thurber,et al.  Parameter estimation and inverse problems , 2005 .

[5]  M. B. Beck,et al.  Uncertainty, identifiability and the propagation of prediction errors: A case study of lake ontario , 1991 .

[6]  Neil McIntyre,et al.  Towards reduced uncertainty in conceptual rainfall‐runoff modelling: dynamic identifiability analysis , 2003 .

[7]  Keith Beven,et al.  A manifesto for the equifinality thesis , 2006 .

[8]  Anthony J. Jakeman,et al.  Ten iterative steps in development and evaluation of environmental models , 2006, Environ. Model. Softw..

[9]  Soroosh Sorooshian,et al.  The Analysis of Structural Identifiability: Theory and Application to Conceptual Rainfall-Runoff Models , 1985 .

[10]  H. Künsch,et al.  Practical identifiability analysis of large environmental simulation models , 2001 .

[11]  A. Holmberg On the practical identifiability of microbial growth models incorporating Michaelis-Menten type nonlinearities , 1982 .

[12]  C. Torrence,et al.  A Practical Guide to Wavelet Analysis. , 1998 .

[13]  Denis Dochain,et al.  Structural Identifiability of Biokinetic Models of Activated-sludge Respiration , 1995 .

[14]  Yves Lecourtier,et al.  Unidentifiable compartmental models: what to do? , 1981 .

[15]  Thorsten Wagener,et al.  Numerical and visual evaluation of hydrological and environmental models using the Monte Carlo analysis toolbox , 2007, Environ. Model. Softw..

[16]  B. Croke,et al.  Addressing ten questions about conceptual rainfall–runoff models with global sensitivity analyses in R , 2013 .

[17]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[18]  Joseph H. A. Guillaume,et al.  Characterising performance of environmental models , 2013, Environ. Model. Softw..

[19]  Ming Ye,et al.  Towards a comprehensive assessment of model structural adequacy , 2012 .

[20]  Johan Karlsson,et al.  Minimal output sets for identifiability. , 2012, Mathematical biosciences.

[21]  M. B. Beck,et al.  Water quality modeling: A review of the analysis of uncertainty , 1987 .

[22]  J. Doherty,et al.  A hybrid regularized inversion methodology for highly parameterized environmental models , 2005 .

[23]  R. Bellman,et al.  On structural identifiability , 1970 .

[24]  H. Pohjanpalo System identifiability based on the power series expansion of the solution , 1978 .

[25]  C. Cobelli,et al.  Parameter and structural identifiability concepts and ambiguities: a critical review and analysis. , 1980, The American journal of physiology.

[26]  Karel J. Keesman,et al.  Optimal parametric sensitivity control for a FED batch reactor , 2001 .

[27]  Maria Pia Saccomani,et al.  DAISY: A new software tool to test global identifiability of biological and physiological systems , 2007, Comput. Methods Programs Biomed..

[28]  Bruno Sudret,et al.  Global sensitivity analysis using polynomial chaos expansions , 2008, Reliab. Eng. Syst. Saf..

[29]  Karel J. Keesman,et al.  Sensitivity analysis in oxidation ditch modelling: the effect of variations in stoichiometric, kinetic and operating parameters on the performance indices , 2001 .

[30]  S. Sorooshian,et al.  Automatic calibration of conceptual rainfall-runoff models: The question of parameter observability and uniqueness , 1983 .

[31]  S. Sorooshian,et al.  Uniqueness and observability of conceptual rainfall‐runoff model parameters: The percolation process examined , 1983 .

[32]  Henrik Madsen,et al.  Automatic calibration of a conceptual rainfall-runoff model using multiple objectives. , 2000 .

[33]  Patrick M. Reed,et al.  Time‐varying sensitivity analysis clarifies the effects of watershed model formulation on model behavior , 2013 .

[34]  Cajo J. F. ter Braak,et al.  Treatment of input uncertainty in hydrologic modeling: Doing hydrology backward with Markov chain Monte Carlo simulation , 2008 .

[35]  Eric Walter,et al.  Identification of Parametric Models: from Experimental Data , 1997 .

[36]  Stefano Marsili-Libelli,et al.  Identification of dynamic models for horizontal subsurface constructed wetlands , 2005 .

[37]  Anthony J. Jakeman,et al.  Selecting among five common modelling approaches for integrated environmental assessment and management , 2013, Environ. Model. Softw..

[38]  Thibault Mathevet,et al.  Dynamic averaging of rainfall‐runoff model simulations from complementary model parameterizations , 2006 .

[39]  S. Sorooshian,et al.  Automatic calibration of conceptual rainfall-runoff models: sensitivity to calibration data , 1996 .

[40]  M. B. Beck,et al.  Accounting for structural error and uncertainty in a model: An approach based on model parameters as stochastic processes , 2012, Environ. Model. Softw..

[41]  Derek J. Pike,et al.  Empirical Model‐building and Response Surfaces. , 1988 .

[42]  Stefano Marsili-Libelli Fuzzy pattern recognition of circadian cycles in ecosystems , 2004 .

[43]  J. D. Stigter,et al.  Optimal parametric sensitivity control for the estimation of kinetic parameters in bioreactors. , 2002, Mathematical biosciences.

[44]  J. Doherty,et al.  Uncertainty assessment and implications for data acquisition in support of integrated hydrologic models , 2012 .

[45]  S. Marsili-Libelli,et al.  Confidence regions of estimated parameters for ecological systems , 2003 .

[46]  A. Jakeman,et al.  Complexity and Integrated Resources Management , 2004 .

[47]  Dongbin Xiu,et al.  High-Order Collocation Methods for Differential Equations with Random Inputs , 2005, SIAM J. Sci. Comput..

[48]  George Kuczera,et al.  Calibration of conceptual hydrological models revisited: 2. Improving optimisation and analysis , 2006 .

[49]  J. Willems,et al.  Parametrizations of linear dynamical systems: Canonical forms and identifiability , 1974 .

[50]  Peter A. Vanrolleghem,et al.  Practical Identifiability of a Biokinetic Model of Activated-sludge Respiration , 1995 .

[51]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[52]  George A. Bekey,et al.  System identification- an introduction and a survey , 1970 .

[53]  J. D. Stigter,et al.  Optimal parametric sensitivity control for a FED batch reactor , 2001, 2001 European Control Conference (ECC).

[54]  Denis Dochain,et al.  Dynamical modelling and estimation in wastewater treatment processes. , 2015 .

[55]  S. Marsili-Libelli,et al.  Accuracy analysis of a respirometer for activated sludge dynamic modelling. , 2002, Water research.

[56]  Circadian Patterns Recognition in Ecosystems by Wavelet Filtering and Fuzzy Clustering , 2004 .