Informal likelihood measures in model assessment: Theoretic development and investigation

Abstract Within hydrology performance criteria such as the Nash–Sutcliffe efficiency have been used to condition the parameter space of a model. Their use is motivated by the fact that the stochastic error series between a model output and corresponding observations is the result of the composite effect of multiple error sources which cannot be described, even in form, a priori. This paper formalises the use of such performance criteria within a Bayesian framework, such as Generalised Likelihood Uncertainty Estimation (GLUE), by introducing the concept of Informal Likelihoods. Informal Likelihoods are used to characterise desirable features in the relationship between the model output and corresponding observed data. A number of common performance criteria are considered as Informal Likelihoods. Analytical results and a simulation indicate all of the performance criteria considered as Informal Likelihoods in this paper have one or more properties which may be considered undesirable, but may perform no less well in conditioning model parameters than formal likelihoods for which the assumptions are only mildly incorrect.

[1]  Ashish Sharma,et al.  Mitigating parameter bias in hydrological modelling due to uncertainty in covariates , 2007 .

[2]  K. Beven.  Towards a coherent philosophy for modelling the environment , 2002, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[3]  Keith Beven,et al.  On constraining TOPMODEL hydrograph simulations using partial saturated area information , 2002 .

[4]  Soroosh Sorooshian,et al.  Toward improved identifiability of hydrologic model parameters: The information content of experimental data , 2002 .

[5]  S. Sorooshian,et al.  Effective and efficient global optimization for conceptual rainfall‐runoff models , 1992 .

[6]  Keith Beven,et al.  A manifesto for the equifinality thesis , 2006 .

[7]  Ezio Todini,et al.  Coherence of the statistical inference process in hydrological forecasting , 2004 .

[8]  L. Joseph,et al.  Bayesian Statistics: An Introduction , 1989 .

[9]  T. McMahon,et al.  Application of the daily rainfall-runoff model MODHYDROLOG to 28 Australian catchments , 1994 .

[10]  J. Nash,et al.  River flow forecasting through conceptual models part I — A discussion of principles☆ , 1970 .

[11]  Soroosh Sorooshian,et al.  Multi-objective global optimization for hydrologic models , 1998 .

[12]  Soroosh Sorooshian,et al.  A framework for development and application of hydrological models , 2001, Hydrology and Earth System Sciences.

[13]  P. Mantovan,et al.  Hydrological forecasting uncertainty assessment: Incoherence of the GLUE methodology , 2006 .

[14]  D. Legates,et al.  Evaluating the use of “goodness‐of‐fit” Measures in hydrologic and hydroclimatic model validation , 1999 .

[15]  A. Petersen-Øverleir.  Accounting for heteroscedasticity in rating curve estimates , 2004 .

[16]  A. H. Murphy,et al.  Skill Scores Based on the Mean Square Error and Their Relationships to the Correlation Coefficient , 1988 .

[17]  Soroosh Sorooshian,et al.  Reply to comment by K. Beven and P. Young on “Bayesian recursive parameter estimation for hydrologic models” , 2003 .

[18]  Keith Beven,et al.  Dalton Medal Lecture: How far can we go in distributed hydrological modelling? , 2001 .

[19]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1951 .

[20]  Keith Beven,et al.  Modelling hydrologic responses in a small forested catchment (Panola Mountain, Georgia, USA): a comparison of the original and a new dynamic TOPMODEL , 2003 .

[21]  Soroosh Sorooshian,et al.  The role of hydrograph indices in parameter estimation of rainfall–runoff models , 2005 .

[22]  G. Kuczera.  Improved parameter inference in catchment models: 1. Evaluating parameter uncertainty , 1983 .

[23]  Kuolin Hsu,et al.  Uncertainty assessment of hydrologic model states and parameters: Sequential data assimilation using the particle filter , 2005 .

[24]  J. Doherty,et al.  METHODOLOGIES FOR CALIBRATION AND PREDICTIVE ANALYSIS OF A WATERSHED MODEL 1 , 2003 .

[25]  R. T. Clarke,et al.  The use of Bayesian methods for fitting rating curves, with case studies , 2005 .

[26]  P. Bates,et al.  Model Validation - Perspectives in Hydrological Science , 2001 .

[27]  Soroosh Sorooshian,et al.  Application of temporal streamflow descriptors in hydrologic model parameter estimation , 2005 .

[28]  J. Nash,et al.  A criterion of efficiency for rainfall-runoff models , 1978 .

[29]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[30]  Keith Beven,et al.  The future of distributed models: model calibration and uncertainty prediction. , 1992 .

[31]  Neil McIntyre,et al.  Towards reduced uncertainty in conceptual rainfall‐runoff modelling: dynamic identifiability analysis , 2003 .

[32]  George Kuczera,et al.  Monte Carlo assessment of parameter uncertainty in conceptual catchment models: the Metropolis algorithm , 1998 .

[33]  George Kuczera,et al.  Semidistributed hydrological modeling: A “saturation path” perspective on TOPMODEL and VIC , 2003 .

[34]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[35]  S. Sorooshian,et al.  Effective and efficient algorithm for multiobjective optimization of hydrologic models , 2003 .

[36]  Keith Beven,et al.  Hydrological processes—Letters. Topographic controls on subsurface storm flow at the hillslope scale for two hydrologically distinct small catchmetns , 1997 .

[37]  Robert E. Davis,et al.  Statistics for the evaluation and comparison of models , 1985 .

[38]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[39]  Keith Beven,et al.  So just why would a modeller choose to be incoherent , 2008 .

[40]  J. McDonnell,et al.  Constraining dynamic TOPMODEL responses for imprecise water table information using fuzzy rule based performance measures , 2004 .

[41]  M. Trosset,et al.  Bayesian recursive parameter estimation for hydrologic models , 2001 .

[42]  Dominic Welsh,et al.  Probability: An Introduction , 1986 .

[43]  Peter C. Young,et al.  Comment on “Bayesian recursive parameter estimation for hydrologic models” by M. Thiemann, M. Trosset, H. Gupta, and S. Sorooshian , 2003 .

[44]  Reginald W. Herschy,et al.  Hydrometry: Principles and Practices , 1978 .

[45]  Soroosh Sorooshian,et al.  Dual state-parameter estimation of hydrological models using ensemble Kalman filter , 2005 .

[46]  S. Chib.  Marginal Likelihood from the Gibbs Output , 1995 .

[47]  Soroosh Sorooshian,et al.  Toward improved calibration of hydrologic models: Multiple and noncommensurable measures of information , 1998 .

[48]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[49]  Todini,et al.  Coupling meteorological and hydrological models for flood forecasting , 2005 .

[50]  S. Sorooshian,et al.  Calibration of watershed models , 2003 .