Model Calibration and Uncertainty Estimation

All rainfall-runoff models are, by definition, simplifications of the real-world system under investigation. The model components are aggregated descriptions of real-world hydrologic processes. One consequence of this is that the model parameters often do not represent directly measurable entities, but must be estimated using measurements of the system response through a process known as model calibration. The objective of this calibration process is to obtain a model with the following characteristics: (i) the input-state-output behavior of the model is consistent with the measurements of catchment behavior, (ii) the model predictions are accurate (i.e. they have negligible bias) and precise (i.e. the prediction uncertainty is relatively small), and (iii) the model structure and behavior are consistent with current hydrologic understanding of reality. This article describes the historic development leading to current views on model calibration, and the algorithms and techniques that have been developed for estimating parameters, thereby enabling the model to mimic the behavior of the hydrologic system. Manual techniques as well as automatic algorithms are addressed. The automatic approaches range from purely random techniques, to local and global search algorithms. An overview of multiobjective and recursive algorithms is also presented. Although it would be desirable to reduce the total output prediction error to zero (i.e. the difference between observed and simulated system behavior) this is generally impossible owing to the unavoidable uncertainties inherent in any rainfall-runoff modeling procedure. These uncertainties stem mainly from the inability of calibration procedures to uniquely identify a single optimal parameter set, from measurement errors associated with the system input and output, and from model structural errors arising from the aggregation of real-world processes into a mathematical model. Some commonly used approaches to estimate these uncertainties and their impacts on the model predictions are discussed. The article ends with a brief discussion about the current status of calibration and how well we are able to represent the effects of uncertainty in the modeling process, and some potential directions.

[1]  H. H. Rosenbrock,et al.  An Automatic Method for Finding the Greatest or Least Value of a Function , 1960, Comput. J..

[2]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[3]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[4]  R. Ibbitt,et al.  Systematic parameter fitting for conceptual models of catchment hydrology , 1970 .

[5]  J. Nash,et al.  River flow forecasting through conceptual models part I — A discussion of principles☆ , 1970 .

[6]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[7]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[8]  P. R. Johnston,et al.  Parameter optimization for watershed models , 1976 .

[9]  G. Pickup TESTING THE EFFICIENCY OF ALGORITHMS AND STRATEGIES FOR AUTOMATIC CALIBRATION OF RAINFALL-RUNOFF MODELS , 1977 .

[10]  J. W. Akitt Function Minimisation Using the Nelder and Mead Simplex Method with Limited Arithmetic Precision: The Self Regenerative Simplex , 1977, Comput. J..

[11]  T. Brubaker,et al.  Nonlinear Parameter Estimation , 1979 .

[12]  S. Sorooshian,et al.  Stochastic parameter estimation procedures for hydrologie rainfall‐runoff models: Correlated and heteroscedastic error cases , 1980 .

[13]  R. Spear Eutrophication in peel inlet—II. Identification of critical uncertainties via generalized sensitivity analysis , 1980 .

[14]  Peter K. Kitanidis,et al.  Real‐time forecasting with a conceptual hydrologic model: 1. Analysis of uncertainty , 1980 .

[15]  G. Bekey,et al.  A global optimization algorithm using adaptive random search , 1980 .

[16]  P. Kitanidis,et al.  Real‐time forecasting with a conceptual hydrologic model: 2. Applications and results , 1980 .

[17]  Stephen J. Burges,et al.  Approximate Error Bounds for Simulated Hydrographs , 1981 .

[18]  G. Hornberger,et al.  Approach to the preliminary analysis of environmental systems , 1981 .

[19]  Emilio Rosenblueth,et al.  Two-point estimates in probabilities , 1981 .

[20]  K. Beven,et al.  ASSESSING THE EFFECT OF SPATIAL PATTERN OF PRECIPITATION IN MODELING STREAM FLOW HYDROGRAPHS , 1982 .

[21]  S. Sorooshian,et al.  Evaluation of Maximum Likelihood Parameter estimation techniques for conceptual rainfall‐runoff models: Influence of calibration data variability and length on model credibility , 1983 .

[22]  G. Kuczera Improved parameter inference in catchment models: 1. Evaluating parameter uncertainty , 1983 .

[23]  D. H. Pilgrim,et al.  Some problems in transferring hydrological relationships between small and large drainage basins and between regions , 1983 .

[24]  Brent M. Troutman,et al.  Runoff prediction errors and bias in parameter estimation induced by spatial variability of precipitation , 1983 .

[25]  G. Kuczera Improved parameter inference in catchment models: 2. Combining different kinds of hydrologic data and testing their compatibility , 1983 .

[26]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[27]  K. Beven,et al.  Shenandoah Watershed Study: Calibration of a Topography‐Based, Variable Contributing Area Hydrological Model to a Small Forested Catchment , 1985 .

[28]  Soroosh Sorooshian,et al.  The Analysis of Structural Identifiability: Theory and Application to Conceptual Rainfall-Runoff Models , 1985 .

[29]  Soroosh Sorooshian,et al.  The relationship between data and the precision of parameter estimates of hydrologic models , 1985 .

[30]  Keith Beven,et al.  Sensitivity analysis, calibration and predictive uncertainty of the Institute of Hydrology Distributed Model , 1985 .

[31]  M. B. Beck,et al.  The identification of conceptual hydrological models for surface water acidification , 1986 .

[32]  Witold F. Krajewski,et al.  Optimization of Complex Hydrologic Models Using Random Search Methods , 1987 .

[33]  M. B. Beck,et al.  Water quality modeling: A review of the analysis of uncertainty , 1987 .

[34]  W. Price Global optimization algorithms for a CAD workstation , 1987 .

[35]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[36]  Larry Brazil,et al.  Multilevel calibration strategy for complex hydrologic simulation models , 1988 .

[37]  Jene Diane Hendrickson,et al.  Comparison of Newton-type and direct search algorithms for calibration of conceptual rainfall-runoff models , 1988 .

[38]  George Kuczera,et al.  On the validity of first-order prediction limits for conceptual hydrologic models , 1988 .

[39]  Keith Beven,et al.  Changing ideas in hydrology — The case of physically-based models , 1989 .

[40]  M. Harr Probabilistic estimates for multivariate analyses , 1989 .

[41]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[42]  J. Harlin Development of a Process Oriented Calibration Scheme for the HBV Hydrological Model , 1991 .

[43]  Keith Beven,et al.  CHANGING RESPONSES IN HYDROLOGY : ASSESSING THE UNCERTAINTY IN PHYSICALLY BASED MODEL PREDICTIONS , 1991 .

[44]  Q. J. Wang The Genetic Algorithm and Its Application to Calibrating Conceptual Rainfall-Runoff Models , 1991 .

[45]  Karel J. Keesman,et al.  Uncertainty propagation and speculation in projective forecasts of environmental change - a lake eutrophication example. , 1991 .

[46]  A. Calver,et al.  Effects of Spatially-Distributed Rainfall on Runoff for a Conceptual Catchment , 1991 .

[47]  C. T. Haan,et al.  Multiobjective Parameter Estimation For Hydrologic Models - Weighting Of Errors , 1991 .

[48]  H. Scholten,et al.  Prediction Uncertainty in an Ecological Model of the Oosterschelde Estuary , 1991 .

[49]  Witold F. Krajewski,et al.  A Monte Carlo Study of rainfall sampling effect on a distributed catchment model , 1991 .

[50]  Keith Beven,et al.  The future of distributed models: model calibration and uncertainty prediction. , 1992 .

[51]  S. Sorooshian,et al.  Effective and efficient global optimization for conceptual rainfall‐runoff models , 1992 .

[52]  Anthony J. Jakeman,et al.  A systematic approach to modelling the dynamic linkage of climate, physical catchment descriptors and hydrologic response components , 1992 .

[53]  C. S. Melching An improved first-order reliability approach for assessing uncertainties in hydrologic modeling , 1992 .

[54]  George Kuczera,et al.  Effect of rainfall errors on accuracy of design flood estimates , 1992 .

[55]  S. Sorooshian,et al.  Shuffled complex evolution approach for effective and efficient global minimization , 1993 .

[56]  Keith Beven,et al.  Prophecy, reality and uncertainty in distributed hydrological modelling , 1993 .

[57]  K. Beven,et al.  Progress and directions in rainfall-runoff modelling , 1993 .

[58]  A. Jakeman,et al.  How much complexity is warranted in a rainfall‐runoff model? , 1993 .

[59]  Soroosh Sorooshian,et al.  Calibration of rainfall‐runoff models: Application of global optimization to the Sacramento Soil Moisture Accounting Model , 1993 .

[60]  Pierre Y. Julien,et al.  Runoff sensitivity to temporal and spatial rainfall variability at runoff plane and small basin scales , 1993 .

[61]  Soroosh Sorooshian,et al.  Optimal use of the SCE-UA global optimization method for calibrating watershed models , 1994 .

[62]  G. Evensen Sequential data assimilation with a nonlinear quasi‐geostrophic model using Monte Carlo methods to forecast error statistics , 1994 .

[63]  Keith Beven,et al.  The sensitivity of hydrological models to spatial rainfall patterns: an evaluation using observed data , 1994 .

[64]  Soroosh Sorooshian,et al.  Effect of rainfall‐sampling errors on simulations of desert flash floods , 1994 .

[65]  Pierre Y. Julien,et al.  Runoff model sensitivity to radar rainfall resolution , 1994 .

[66]  Dong-Jun Seo,et al.  An Intercomparison Study of NEXRAD Precipitation Estimates , 1996 .

[67]  P. Young,et al.  Simplicity out of complexity in environmental modelling: Occam's razor revisited. , 1996 .

[68]  J. Stiffler Reliability estimation , 1996 .

[69]  Paul O'Connell,et al.  Modelling the effects of spatial variability in rainfall on catchment response. 2. Experiments with distributed and lumped models , 1996 .

[70]  K. Beven,et al.  Bayesian Estimation of Uncertainty in Runoff Prediction and the Value of Data: An Application of the GLUE Approach , 1996 .

[71]  Jonathan R. M. Hosking,et al.  MODELLING THE EFFECTS OF SPATIAL VARIABILITY IN RAINFALL ON CATCHMENT RESPONSE. : 1. FORMULATION AND CALIBRATION OF A STOCHASTIC RAINFALL FIELD MODEL , 1996 .

[72]  S. Sorooshian,et al.  Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks , 1997 .

[73]  Vijay P. Singh,et al.  Effect of spatial and temporal variability in rainfall and watershed characteristics on stream flow hydrograph , 1997 .

[74]  Dong-Jun Seo,et al.  Space-time scale sensitivity of the Sacramento model to radar-gage precipitation inputs , 1997 .

[75]  K. Beven,et al.  MODELLING THE HYDROLOGICAL RESPONSE OF MEDITERRANEAN CATCHMENTS, PRADES, CATALONIA. THE USE OF DISTRIBUTED MODELS AS AIDS TO HYPOTHESIS FORMULATION , 1997 .

[76]  George Kuczera,et al.  Assessment of hydrologic parameter uncertainty and the worth of multiresponse data , 1998 .

[77]  Soroosh Sorooshian,et al.  Multi-objective global optimization for hydrologic models , 1998 .

[78]  Keith Beven,et al.  Use of spatially distributed water table observations to constrain uncertainty in a rainfall–runoff model , 1998 .

[79]  K. Beven,et al.  On constraining the predictions of a distributed model: The incorporation of fuzzy estimates of saturated areas into the calibration process , 1998 .

[80]  Soroosh Sorooshian,et al.  On the simulation of infiltration‐ and saturation‐excess runoff using radar‐based rainfall estimates: Effects of algorithm uncertainty and pixel aggregation , 1998 .

[81]  Soroosh Sorooshian,et al.  Toward improved calibration of hydrologic models: Multiple and noncommensurable measures of information , 1998 .

[82]  Victor Koren,et al.  Comparing Mean Areal Precipitation Estimates from NEXRAD and Rain Gauge Networks , 1999 .

[83]  D. Legates,et al.  Evaluating the use of “goodness‐of‐fit” Measures in hydrologic and hydroclimatic model validation , 1999 .

[84]  George Kuczera,et al.  Probabilistic optimization for conceptual rainfall-runoff models: A comparison of the shuffled complex evolution and simulated annealing algorithms , 1999 .

[85]  H. Madsen,et al.  Comparison of extended and ensemble Kalman filters for data assimilation in coastal area modelling , 1999 .

[86]  Adrian E. Raftery,et al.  Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors , 1999 .

[87]  S. Uhlenbrook,et al.  Prediction uncertainty of conceptual rainfall-runoff models caused by problems in identifying model parameters and structure , 1999 .

[88]  Dong-Jun Seo,et al.  Scale dependencies of hydrologic models to spatial variability of precipitation , 1999 .

[89]  Keith Beven,et al.  The use of generalised likelihood measures for uncertainty estimation in high order models of environmental systems , 2000 .

[90]  J. Seibert Multi-criteria calibration of a conceptual runoff model using a genetic algorithm , 2000 .

[91]  Witold F. Krajewski,et al.  Evaluating NEXRAD Multisensor Precipitation Estimates for Operational Hydrologic Forecasting , 2000 .

[92]  S. Sorooshian,et al.  Evaluation of PERSIANN system satellite-based estimates of tropical rainfall , 2000 .

[93]  Henrik Madsen,et al.  Automatic calibration of a conceptual rainfall-runoff model using multiple objectives. , 2000 .

[94]  Soroosh Sorooshian,et al.  Toward improved calibration of hydrologic models: Combining the strengths of manual and automatic methods , 2000 .

[95]  Jacques Lavabre,et al.  Impact of imperfect rainfall knowledge on the efficiency and the parameters of watershed models , 2001 .

[96]  Kuolin Hsu,et al.  Diurnal Variability of Tropical Rainfall Retrieved from Combined GOES and TRMM Satellite Information , 2002 .

[97]  Soroosh Sorooshian,et al.  Toward improved identifiability of hydrologic model parameters: The information content of experimental data , 2002 .

[98]  Soroosh Sorooshian,et al.  Toward improved streamflow forecasts: value of semidistributed modeling , 2001 .

[99]  M. Trosset,et al.  Bayesian recursive parameter estimation for hydrologic models , 2001 .

[100]  M. G. Anderson,et al.  DATA-BASED MECHANISTIC MODELLING AND VALIDATION OF RAINFALL-FLOW PROCESSES , 2001 .

[101]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[102]  K. Georgakakos,et al.  On the parametric and NEXRAD-radar sensitivities of a distributed hydrologic model suitable for operational use , 2001 .

[103]  Keith Beven,et al.  Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the GLUE methodology , 2001 .

[104]  U. Shamir,et al.  The characteristic time scale for basin hydrological response using radar data , 2001 .

[105]  Howard S. Wheater,et al.  Estimation and propagation of parametric uncertainty in environmental models , 2002 .

[106]  Keith Beven,et al.  Towards an alternative blueprint for a physically based digitally simulated hydrologic response modelling system , 2002 .

[107]  S. Sorooshian,et al.  A Shuffled Complex Evolution Metropolis algorithm for optimization and uncertainty assessment of hydrologic model parameters , 2002 .

[108]  Henrik Madsen,et al.  Comparison of different automated strategies for calibration of rainfall-runoff models , 2002 .

[109]  Keith Beven,et al.  Testing the distributed water table predictions of TOPMODEL (allowing for uncertainty in model calibration): The death of TOPMODEL? , 2002 .

[110]  Keith Beven,et al.  On constraining TOPMODEL hydrograph simulations using partial saturated area information , 2002 .

[111]  Vijay P. Singh,et al.  A toolkit for the development and application of parsimonious hydrological models. , 2002 .

[112]  Shlomo P. Neuman Accounting for conceptual model uncertainty via maximum likelihood Bayesian model averaging , 2002 .

[113]  D. McLaughlin,et al.  Hydrologic Data Assimilation with the Ensemble Kalman Filter , 2002 .

[114]  Feyzan Misirli Baysal Improving efficiency and effectiveness of Bayesian recursive parameter estimation for hydrologic models , 2003 .

[115]  Neil McIntyre,et al.  Towards reduced uncertainty in conceptual rainfall‐runoff modelling: dynamic identifiability analysis , 2003 .

[116]  Soroosh Sorooshian,et al.  Reply to comment by K. Beven and P. Young on “Bayesian recursive parameter estimation for hydrologic models” , 2003 .

[117]  S. Sorooshian,et al.  Calibration of watershed models , 2003 .

[118]  Peter C. Young,et al.  Comment on “Bayesian recursive parameter estimation for hydrologic models” by M. Thiemann, M. Trosset, H. Gupta, and S. Sorooshian , 2003 .

[119]  S. Sorooshian,et al.  Effective and efficient algorithm for multiobjective optimization of hydrologic models , 2003 .

[120]  Thorsten Wagener Evaluation of catchment models , 2003 .

[121]  Victor Koren,et al.  Runoff response to spatial variability in precipitation: an analysis of observed data , 2004 .

[122]  M. J. Hall,et al.  Rainfall-Runoff Modelling , 2004 .

[123]  Konstantine P. Georgakakos,et al.  The distributed model intercomparison project (DMIP) , 2004 .

[124]  Soroosh Sorooshian,et al.  A hydroarchive for the free exchange of hydrological software , 2004 .

[125]  Matthias Ehrgott,et al.  Multicriteria Optimization , 2005 .

[126]  Keith Beven,et al.  Data‐based modelling of runoff and chemical tracer concentrations in the Haute‐Mentue research catchment (Switzerland) , 2005 .

[127]  Keith Beven,et al.  A manifesto for the equifinality thesis , 2006 .