Two statistics for evaluating parameter identifiability and error reduction

Summary Two statistics are presented that can be used to rank input parameters utilized by a model in terms of their relative identifiability based on a given or possible future calibration dataset. Identifiability is defined here as the capability of model calibration to constrain parameters used by a model. Both statistics require that the sensitivity of each model parameter be calculated for each model output for which there are actual or presumed field measurements. Singular value decomposition (SVD) of the weighted sensitivity matrix is then undertaken to quantify the relation between the parameters and observations that, in turn, allows selection of calibration solution and null spaces spanned by unit orthogonal vectors. The first statistic presented, “parameter identifiability”, is quantitatively defined as the direction cosine between a parameter and its projection onto the calibration solution space. This varies between zero and one, with zero indicating complete non-identifiability and one indicating complete identifiability. The second statistic, “relative error reduction”, indicates the extent to which the calibration process reduces error in estimation of a parameter from its pre-calibration level where its value must be assigned purely on the basis of prior expert knowledge. This is more sophisticated than identifiability, in that it takes greater account of the noise associated with the calibration dataset. Like identifiability, it has a maximum value of one (which can only be achieved if there is no measurement noise). Conceptually it can fall to zero; and even below zero if a calibration problem is poorly posed. An example, based on a coupled groundwater/surface-water model, is included that demonstrates the utility of the statistics.

[1]  Carlos E. Ruiz,et al.  Using spatially distributed parameters and multi‐response objective functions to solve parameterization of complex applications of semi‐distributed hydrological models , 2008 .

[2]  D. E. Prudic,et al.  GSFLOW - Coupled Ground-Water and Surface-Water Flow Model Based on the Integration of the Precipitation-Runoff Modeling System (PRMS) and the Modular Ground-Water Flow Model (MODFLOW-2005) , 2008 .

[3]  R. Hunt,et al.  Are Models Too Simple? Arguments for Increased Parameterization , 2007, Ground water.

[4]  Soroosh Sorooshian,et al.  Toward improved calibration of hydrologic models: Combining the strengths of manual and automatic methods , 2000 .

[5]  A. Jakeman,et al.  How much complexity is warranted in a rainfall‐runoff model? , 1993 .

[6]  C. Tiedeman,et al.  Effective Groundwater Model Calibration: With Analysis of Data, Sensitivities, Predictions, and Uncertainty , 2007 .

[7]  G. H. Leavesley,et al.  Precipitation-runoff modeling system; user's manual , 1983 .

[8]  Neil McIntyre,et al.  Towards reduced uncertainty in conceptual rainfall‐runoff modelling: dynamic identifiability analysis , 2003 .

[9]  John Doherty,et al.  An advanced regularization methodology for use in watershed model calibration , 2006 .

[10]  Keith Beven,et al.  The future of distributed models: model calibration and uncertainty prediction. , 1992 .

[11]  Clifford H. Thurber,et al.  Parameter estimation and inverse problems , 2005 .

[12]  M. B. Beck,et al.  Uncertainty, identifiability and the propagation of prediction errors: A case study of lake ontario , 1991 .

[13]  W. Yeh.  Review of Parameter Identification Procedures in Groundwater Hydrology: The Inverse Problem , 1986 .

[14]  J. Doherty,et al.  Role of the calibration process in reducing model predictive error , 2005 .

[15]  Mary P Anderson,et al.  Importance of Unsaturated Zone Flow for Simulating Recharge in a Humid Climate , 2008, Ground water.

[16]  R. Hunt,et al.  The importance of diverse data types to calibrate a watershed model of the Trout Lake Basin, Northern Wisconsin, USA , 2006 .

[17]  W. Menke.  Geophysical data analysis : discrete inverse theory , 1984 .

[18]  Mary C. Hill,et al.  UCODE_2005 and six other computer codes for universal sensitivity analysis, calibration, and uncertainty evaluation constructed using the JUPITER API , 2006 .

[19]  J. Doherty,et al.  METHODOLOGIES FOR CALIBRATION AND PREDICTIVE ANALYSIS OF A WATERSHED MODEL 1 , 2003 .

[20]  Soroosh Sorooshian,et al.  Toward improved identifiability of hydrologic model parameters: The information content of experimental data , 2002 .

[21]  Keith Beven,et al.  Changing ideas in hydrology — The case of physically-based models , 1989 .

[22]  S. Sorooshian,et al.  Automatic calibration of conceptual rainfall-runoff models: The question of parameter observability and uniqueness , 1983 .

[23]  T. Bullen,et al.  Trout Lake, Wisconsin: A water, energy, and biogeochemical budgets program site , 2000 .

[24]  Soroosh Sorooshian,et al.  Sensitivity analysis of a land surface scheme using multicriteria methods , 1999 .

[25]  Keith Beven,et al.  Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the GLUE methodology , 2001 .