Evaluation of statistical models for forecast errors from the HBV model

Summary Three statistical models for the forecast errors for inflow into the Langvatn reservoir in Northern Norway have been constructed and tested according to the agreement between (i) the forecast distribution and the observations and (ii) median values of the forecast distribution and the observations. For the first model observed and forecasted inflows were transformed by the Box–Cox transformation before a first order auto-regressive model was constructed for the forecast errors. The parameters were conditioned on weather classes. In the second model the Normal Quantile Transformation (NQT) was applied on observed and forecasted inflows before a similar first order auto-regressive model was constructed for the forecast errors. For the third model positive and negative errors were modeled separately. The errors were first NQT-transformed before conditioning the mean error values on climate, forecasted inflow and yesterday’s error. To test the three models we applied three criterions: we wanted (a) the forecast distribution to be reliable; (b) the forecast intervals to be narrow; (c) the median values of the forecast distribution to be close to the observed values. Models 1 and 2 gave almost identical results. The median values improved the forecast with Nash–Sutcliffe Reff increasing from 0.77 for the original forecast to 0.87 for the corrected forecasts. Models 1 and 2 over-estimated the forecast intervals but gave the narrowest intervals. Their main drawback was that the distributions are less reliable than Model 3. For Model 3 the median values did not fit well since the auto-correlation was not accounted for. Since Model 3 did not benefit from the potential variance reduction that lies in bias estimation and removal it gave on average wider forecasts intervals than the two other models. At the same time Model 3 on average slightly under-estimated the forecast intervals, probably explained by the use of average measures to evaluate the fit.

[1]  G. Kuczera.  Improved parameter inference in catchment models: 1. Evaluating parameter uncertainty , 1983 .

[2]  Mario L. V. Martina,et al.  A Bayesian decision approach to rainfall thresholds based flood warning , 2005 .

[3]  George Kuczera,et al.  Critical evaluation of parameter consistency and predictive uncertainty in hydrological modeling: A case study using Bayesian total error analysis , 2009 .

[4]  Alberto Montanari,et al.  Estimating the uncertainty of hydrological forecasts: A statistical approach , 2008 .

[5]  A. Raftery,et al.  Probabilistic forecasts, calibration and sharpness , 2007 .

[6]  Soroosh Sorooshian,et al.  Multi-objective global optimization for hydrologic models , 1998 .

[7]  André Berchtold,et al.  Mixture transition distribution (MTD) modeling of heteroscedastic time series , 2003, Comput. Stat. Data Anal..

[8]  George Kuczera,et al.  Bayesian analysis of input uncertainty in hydrological modeling: 2. Application , 2006 .

[9]  Soroosh Sorooshian,et al.  Reply to comment by K. Beven and P. Young on “Bayesian recursive parameter estimation for hydrologic models” , 2003 .

[10]  K. Beven.  On undermining the science? , 2006 .

[11]  John Bjørnar Bremnes,et al.  Probabilistic Forecasts of Precipitation in Terms of Quantiles Using NWP Model Output , 2004 .

[12]  Roman Krzysztofowicz,et al.  Bayesian system for probabilistic river stage forecasting , 2002 .

[13]  Soroosh Sorooshian,et al.  Toward improved calibration of hydrologic models: Multiple and noncommensurable measures of information , 1998 .

[14]  Jens Christian Refsgaard,et al.  Construction, Calibration And Validation of Hydrological Models , 1990 .

[15]  P. Reichert,et al.  Hydrological modelling of the Chaohe Basin in China: Statistical model formulation and Bayesian inference , 2007 .

[16]  Keith Beven,et al.  So just why would a modeller choose to be incoherent , 2008 .

[17]  D. Kavetski,et al.  Towards a Bayesian total error analysis of conceptual rainfall-runoff models: Characterising model error using storm-dependent parameters , 2006 .

[18]  Keith Beven,et al.  The future of distributed models: model calibration and uncertainty prediction. , 1992 .

[19]  Alberto Montanari,et al.  Large sample behaviors of the generalized likelihood uncertainty estimation (GLUE) in assessing the uncertainty of rainfall‐runoff simulations , 2005 .

[20]  Soroosh Sorooshian,et al.  A framework for development and application of hydrological models , 2001, Hydrology and Earth System Sciences.

[21]  V. Singh,et al.  Computer Models of Watershed Hydrology , 1995 .

[22]  Ezio Todini.  Using phase-state modelling for inferring forecasting uncertainty in nonlinear stochastic decision schemes , 1999 .

[23]  Asaad Y. Shamseldin,et al.  Development of a possibilistic method for the evaluation of predictive uncertainty in rainfall‐runoff modeling , 2007 .

[24]  F. Atger,et al.  The Skill of Ensemble Prediction Systems , 1999 .

[25]  Kolbjørn Engeland,et al.  Assessing uncertainties in a conceptual water balance model using Bayesian methodology / Estimation bayésienne des incertitudes au sein d’une modélisation conceptuelle de bilan hydrologique , 2005 .

[26]  George Kuczera,et al.  Bayesian analysis of input uncertainty in hydrological modeling: 1. Theory , 2006 .

[27]  Ezio Todini,et al.  Comment on: ‘On undermining the science?’ by Keith Beven , 2007 .

[28]  Nong Shang,et al.  Parameter uncertainty and interaction in complex environmental models , 1994 .

[29]  George Kuczera,et al.  Understanding predictive uncertainty in hydrologic modeling: The challenge of identifying input and structural errors , 2010 .

[30]  Cajo J. F. ter Braak,et al.  Treatment of input uncertainty in hydrologic modeling: Doing hydrology backward with Markov chain Monte Carlo simulation , 2008 .

[31]  Mario L. V. Martina,et al.  Reply to comment by Keith Beven, Paul Smith and Jim Freer on “Hydrological forecasting uncertainty assessment: Incoherence of the GLUE methodology” , 2007 .

[32]  P. Reed,et al.  Hydrology and Earth System Sciences Discussions Comparing Sensitivity Analysis Methods to Advance Lumped Watershed Model Identification and Evaluation , 2022 .

[33]  L. Zadeh.  Fuzzy sets as a basis for a theory of possibility , 1999 .

[34]  George Kuczera,et al.  Monte Carlo assessment of parameter uncertainty in conceptual catchment models: the Metropolis algorithm , 1998 .

[35]  M. Trosset,et al.  Bayesian recursive parameter estimation for hydrologic models , 2001 .

[36]  Didier Dubois,et al.  Possibility theory and its applications: a retrospective and prospective view , 2003, The 12th IEEE International Conference on Fuzzy Systems, 2003. FUZZ '03..

[37]  P. Mantovan,et al.  Hydrological forecasting uncertainty assessment: Incoherence of the GLUE methodology , 2006 .

[38]  Stefania Tamea,et al.  Verification tools for probabilistic forecasts of continuous hydrological variables , 2006 .

[39]  Peter C. Young,et al.  Comment on “Bayesian recursive parameter estimation for hydrologic models” by M. Thiemann, M. Trosset, H. Gupta, and S. Sorooshian , 2003 .

[40]  V. Singh,et al.  The HBV model. , 1995 .