Alternative configurations of quantile regression for estimating predictive uncertainty in water level forecasts for the upper Severn River: A comparison

The present study comprises an intercomparison of different configurations of a statistical post-processor that is used to estimate predictive hydrological uncertainty. It builds on earlier work by Weerts, Winsemius and Verkade (2011; hereafter referred to as WWV2011), who used the quantile regression technique to estimate predictive hydrological uncertainty using a deterministic water level forecast as a predictor. The various configurations are designed to address two issues with the WWV2011 implementation: (i) quantile crossing, which causes non-strictly rising cumulative predictive distributions, and (ii) the use of linear quantile models to describe joint distributions that may not be strictly linear. Thus, four configurations were built: (i) a ''classical" quantile regression, (ii) a configuration that implements a non-crossing quantile technique, (iii) a configuration where quantile models are built in normal space after application of the normal quantile transformation (NQT) (similar to the implementation used by WWV2011), and (iv) a configuration that builds quantile model separately on separate domains of the predictor. Using each configuration, four reforecasting series of water levels at 14 stations in the upper Severn River were established. The quality of these four series was intercompared using a set of graphical and numerical verification metrics. Intercomparison showed that reliability and sharpness vary across configurations, but in none of the configurations do these two forecast quality aspects improve simultaneously. Further analysis shows that skills in terms of the Brier skill score, mean continuous ranked probability skill score and relative operating characteristic score is very similar across the four configurations.

[1]  A. H. Murphy,et al.  What Is a Good Forecast? An Essay on the Nature of Goodness in Weather Forecasting , 1993 .

[2]  Florian Pappenberger,et al.  Do probabilistic forecasts lead to better decisions , 2012 .

[3]  K. Bogner,et al.  Post‐processing hydrological ensemble predictions intercomparison experiment , 2013 .

[4]  Patrick Willems,et al.  A non-parametric data-based approach for probabilistic flood forecasting in support of uncertainty communication , 2012, Environ. Model. Softw..

[5]  J. S. Verkade,et al.  Estimation of Predictive Hydrological Uncertainty using Quantile Regression , 2010 .

[6]  W. Briggs.  Statistical Methods in the Atmospheric Sciences , 2007 .

[7]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[8]  Jaap Schellekens,et al.  The Delft-FEWS flow forecasting system , 2013, Environ. Model. Softw..

[9]  Dimitri Solomatine,et al.  A novel method to estimate model uncertainty using machine learning techniques , 2009 .

[10]  R. Koenker.  Quantile Regression: Name Index , 2005 .

[11]  Yuqiong Liu,et al.  The Ensemble Verification System (EVS): A software tool for verifying ensemble forecasts of hydrometeorological and hydrologic variables at discrete locations , 2010, Environ. Model. Softw..

[12]  Joseph P. Romano,et al.  The stationary bootstrap , 1994 .

[13]  T. A. Brown.  Admissible Scoring Systems for Continuous Distributions. , 1974 .

[14]  G. Brier.  VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[15]  Paolo Reggiani,et al.  Uncertainty assessment via Bayesian revision of ensemble streamflow predictions in the operational river Rhine forecasting system , 2009 .

[16]  J. A. Vrugt,et al.  Hydrology and Earth System Sciences Discussions , 2022 .

[17]  Florian Pappenberger,et al.  Multiscale error analysis, correction, and predictive uncertainty estimation in a flood forecasting system , 2011 .

[18]  Roman Krzysztofowicz,et al.  The case for probabilistic forecasting in hydrology , 2001 .

[19]  Keith Beven,et al.  Forecasting flash floods using data-based mechanistic models and NORA radar rainfall forecasts , 2014 .

[20]  J. S. Verkade,et al.  Estimating the benefits of single value and probability forecasting for flood warning , 2011 .

[21]  D. Solomatine,et al.  Predicting model uncertainty using quantile regression and 1 UNEEC methods and their comparison on contrasting 2 catchments 3 4 , 2015 .

[22]  Dong-Jun Seo,et al.  Short-term ensemble streamflow forecasting using operationally-produced single-valued streamflow forecasts , 2010 .

[23]  John Bjørnar Bremnes,et al.  Probabilistic Forecasts of Precipitation in Terms of Quantiles Using NWP Model Output , 2004 .

[24]  Florian Pappenberger,et al.  Ensemble flood forecasting: a review. , 2009 .

[25]  Dimitri Solomatine,et al.  Estimation of predictive hydrologic uncertainty using the quantile regression and UNEEC methods and their comparison on contrasting catchments , 2014 .

[26]  R. Koenker.  Quantile Regression: Fundamentals of Quantile Regression , 2005 .

[27]  Anton H. Westveld,et al.  Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation , 2005 .

[28]  P. Young,et al.  Risk-based probabilistic fluvial flood forecasting for integrated catchment models - Phase 1 Report , 2009 .

[29]  Henrik Madsen,et al.  Using quantile regression to extend an existing wind power forecasting system with probabilistic forecasts , 2006 .

[30]  Philip H. Ramsey.  Statistical Methods in the Atmospheric Sciences , 2005 .

[31]  Jaap Schellekens,et al.  The use of MOGREPS ensemble rainfall forecasts in operational flood forecasting systems across England and Wales. , 2011 .

[32]  Ezio Todini,et al.  A model conditional processor to assess predictive uncertainty in flood forecasting , 2008 .

[33]  A. Montanari.  Deseasonalisation of hydrological time series through the normal quantile transform , 2005 .

[34]  H. Bondell,et al.  Noncrossing quantile regression curve estimation. , 2010, Biometrika.

[35]  A. Brath,et al.  A stochastic approach for assessing the uncertainty of rainfall‐runoff simulations , 2004 .

[36]  Adrian E. Raftery,et al.  Bayesian Model Selection in Structural Equation Models , 1992 .

[37]  Albrecht Weerts,et al.  Post-processing ECMWF precipitation and temperature ensemble reforecasts for operational hydrologic forecasting at various spatial scales ☆ , 2013 .

[38]  A. Raftery,et al.  Using Bayesian Model Averaging to Calibrate Forecast Ensembles , 2005 .

[39]  James D. Brown,et al.  The Science of NOAA's Operational Hydrologic Ensemble Forecast Service , 2014 .

[40]  Roman Krzysztofowicz,et al.  Hydrologic uncertainty processor for probabilistic river stage forecasting , 2000 .

[41]  R. L. Winkler,et al.  Scoring Rules for Continuous Probability Distributions , 1976 .

[42]  H. Hersbach.  Decomposition of the Continuous Ranked Probability Score for Ensemble Prediction Systems , 2000 .

[43]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[44]  Murray DaleJon.  Probabilistic flood forecasting and decision-making: an innovative risk-based approach , 2012 .

[45]  I. Jolliffe,et al.  Forecast verification : a practitioner's guide in atmospheric science , 2011 .

[46]  A. Weerts,et al.  Estimation of the uncertainty in water level forecasts at ungauged river locations using quantile regression , 2012 .