Evaluation of a nonparametric post‐processor for bias correction and uncertainty estimation of hydrologic predictions

This paper evaluates a nonparametric technique for estimating the conditional probability distribution of a predictand given a vector of predictors. In the current application, the predictors are formed from a multimodel ensemble of simulated streamflows, such that the hydrologic uncertainties are modelled independently of any forcing uncertainties. The technique is based on Bayesian optimal linear estimation of indicator variables and is analogous to indicator cokriging (ICK) in geostatistics. By developing linear estimators for the conditional probability that the observed variable does not exceed several thresholds, ICK provides a discrete approximation of the full conditional probability distribution. The weights of the predictors can be chosen to minimize the expected error variance at each threshold (the Brier score) or, without loss of analytical tractability, a combination of the error variance and the expected square bias conditional upon the observation, i.e. the Type‐II conditional bias (CB). The latter is referred to as CB‐penalized ICK (CBP‐ICK) and is an important enhancement to ICK. Indeed, the biases in atmospheric and hydrologic predictions generally increase towards the tails of their probability distributions. The performance of CBP‐ICK is evaluated for selected basins in the eastern USA using a range of probabilistic verification metrics and associated confidence intervals for the sampling uncertainties. Overall, CBP‐ICK produces unbiased and skillful estimates of the hydrologic uncertainties, with some sensitivity to the calibration data period at high flow thresholds. More generally, we argue that the common aim in statistical post‐processing of ‘maximizing sharpness subject to reliability (Type‐I CB)’ should be recast to accommodate both the Type‐I and Type‐II CBs, as both are important for practical applications of hydrologic predictions. Copyright © 2012 John Wiley & Sons, Ltd.

[1]  G. Brier VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[2]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[3]  A. H. Murphy,et al.  A General Framework for Forecast Verification , 1987 .

[4]  R. Ferguson Accuracy and precision of methods for estimating river loads , 1987 .

[5]  J. Hanley The Robustness of the "Binormal" Assumptions Used in Fitting ROC Curves , 1988, Medical decision making : an international journal of the Society for Medical Decision Making.

[6]  Joseph P. Romano,et al.  The stationary bootstrap , 1994 .

[7]  Dong-Jun Seo,et al.  Nonlinear estimation of spatial distribution of rainfall — An indicator cokriging approach , 1996 .

[8]  Roman Krzysztofowicz,et al.  A bivariate meta-Gaussian density for use in hydrology , 1997 .

[9]  Roman Krzysztofowicz,et al.  Bayesian theory of probabilistic forecasting via deterministic hydrologic model , 1999 .

[10]  C. Metz,et al.  "Proper" Binormal ROC Curves: Theory and Maximum-Likelihood Estimation. , 1999, Journal of mathematical psychology.

[11]  Witold F. Krajewski,et al.  Conditional Bias in Radar Rainfall Estimation. , 2000 .

[12]  N. Graham,et al.  Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation , 2002 .

[13]  Roman Krzysztofowicz,et al.  Bayesian system for probabilistic river stage forecasting , 2002 .

[14]  Tempei Hashino,et al.  Distributions-oriented verification of probability forecasts for small data samples , 2003 .

[15]  Roman Krzysztofowicz,et al.  Hydrologic uncertainty processor for probabilistic stage transition forecasting , 2004 .

[16]  Tempei Hashino,et al.  Distributions-Oriented Verification of Ensemble Streamflow Predictions , 2004 .

[17]  Alan David Hutson,et al.  Resampling Methods for Dependent Data , 2004, Technometrics.

[18]  A. Raftery,et al.  Using Bayesian Model Averaging to Calibrate Forecast Ensembles , 2005 .

[19]  Anton H. Westveld,et al.  Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation , 2005 .

[20]  J Schaake,et al.  U.S. MOPEX DATA SET , 2006 .

[21]  George H. Leavesley,et al.  The model parameter estimation experiment (MOPEX) , 2006 .

[22]  Tempei Hashino,et al.  Evaluation of bias-correction methods for ensemble streamflow volume forecasts , 2006 .

[23]  J. Schaake,et al.  A statistical post-processor for accounting of hydrologic uncertainty in short-range ensemble streamflow prediction , 2006 .

[24]  A. Raftery,et al.  Probabilistic forecasts, calibration and sharpness , 2007 .

[25]  Thomas M. Hamill,et al.  Comparison of Ensemble-MOS Methods Using GFS Reforecasts , 2007 .

[26]  Yuqiong Liu,et al.  Uncertainty in hydrologic modeling: Toward an integrated data assimilation framework , 2007 .

[27]  W. Briggs Statistical Methods in the Atmospheric Sciences , 2007 .

[28]  Dimitri P. Solomatine,et al.  Data‐driven approaches for estimating uncertainty in rainfall‐runoff modelling , 2008 .

[29]  K. Bogner,et al.  Error‐correction methods and evaluation of an ensemble based hydrological forecasting system for the Upper Danube catchment , 2008 .

[30]  David S. Richardson,et al.  On the effect of ensemble size on the discrete and continuous ranked probability scores , 2008 .

[31]  Alberto Montanari,et al.  Estimating the uncertainty of hydrological forecasts: A statistical approach , 2008 .

[32]  J. Schaake,et al.  Correcting Errors in Streamflow Forecast Ensemble Mean and Spread , 2008 .

[33]  Ezio Todini,et al.  A model conditional processor to assess predictive uncertainty in flood forecasting , 2008 .

[34]  Vera Pawlowsky-Glahn,et al.  Indicator Kriging without Order Relation Violations , 2008 .

[35]  Paolo Reggiani,et al.  Uncertainty assessment via Bayesian revision of ensemble streamflow predictions in the operational river Rhine forecasting system , 2009 .

[36]  Daniel S. Wilks,et al.  Extending logistic regression to provide full‐probability‐distribution MOS forecasts , 2009 .

[37]  L. Shawn Matott,et al.  Evaluating uncertainty in integrated environmental models: A review of concepts and tools , 2009 .

[38]  Florian Pappenberger,et al.  Ensemble flood forecasting: a review. , 2009 .

[39]  James D. Brown,et al.  Prospects for the open treatment of uncertainty in environmental research , 2010 .

[40]  Dong-Jun Seo,et al.  Short-term ensemble streamflow forecasting using operationally-produced single-valued streamflow forecasts , 2010 .

[41]  Ezio Todini,et al.  Recent developments in predictive uncertainty assessment based on the model conditional processor approach , 2010 .

[42]  Dong-Jun Seo,et al.  A Nonparametric Postprocessor for Bias Correction of Hydrometeorological and Hydrologic Ensemble Forecasts , 2010 .

[43]  Yuqiong Liu,et al.  The Ensemble Verification System (EVS): A software tool for verifying ensemble forecasts of hydrometeorological and hydrologic variables at discrete locations , 2010, Environ. Model. Softw..

[44]  J. S. Verkade,et al.  Estimation of Predictive Hydrological Uncertainty using Quantile Regression , 2010 .

[45]  I. Jolliffe,et al.  Forecast verification : a practitioner's guide in atmospheric science , 2011 .

[46]  Florian Pappenberger,et al.  Multiscale error analysis, correction, and predictive uncertainty estimation in a flood forecasting system , 2011 .

[47]  Aizhong Ye,et al.  A hydrologic post-processor for ensemble streamflow predictions , 2011 .

[48]  Yuqiong Liu,et al.  A wavelet-based approach to assessing timing errors in hydrologic predictions , 2011 .

[49]  D. Seo Conditional bias-penalized kriging (CBPK) , 2012, Stochastic Environmental Research and Risk Assessment.