Improving probabilistic prediction of daily streamflow by identifying Pareto optimal approaches for modeling heteroscedastic residual errors

Reliable and precise probabilistic prediction of daily catchment-scale streamflow requires statistical characterization of residual errors of hydrological models. This study focuses on approaches for representing error heteroscedasticity with respect to simulated streamflow, i.e., the pattern of larger errors in higher streamflow predictions. We evaluate 8 common residual error schemes, including standard and weighted least squares, the Box-Cox transformation (with fixed and calibrated power parameter λ) and the log-sinh transformation. Case studies include 17 perennial and 6 ephemeral catchments in Australia and USA, and two lumped hydrological models. Performance is quantified using predictive reliability, precision and volumetric bias metrics. We find the choice of heteroscedastic error modelling approach significantly impacts on predictive performance, though no single scheme simultaneously optimizes all performance metrics. The set of Pareto optimal schemes, reflecting performance trade-offs, comprises Box-Cox schemes with λ of 0.2 and 0.5, and the log scheme (λ=0, perennial catchments only). These schemes significantly outperform even the average-performing remaining schemes (e.g., across ephemeral catchments, median precision tightens from 105% to 40% of observed streamflow, and median biases decrease from 25% to 4%). Theoretical interpretations of empirical results highlight the importance of capturing the skew/kurtosis of raw residuals and reproducing zero flows. Paradoxically, calibration of λ is often counterproductive: in perennial catchments, it tends to overfit low flows at the expense of abysmal precision in high flows. The log-sinh transformation is dominated by the simpler Pareto optimal schemes listed above. Recommendations for researchers and practitioners seeking robust residual error schemes for practical work are provided. This article is protected by copyright. All rights reserved.

[1]  Martin F. Lambert,et al.  A strategy for diagnosing and interpreting hydrological model nonstationarity , 2014 .

[2]  C. Perrin,et al.  Does a large number of parameters enhance model performance? Comparative assessment of common catchment model structures on 429 catchments , 2001 .

[3]  G. Kuczera.  Improved parameter inference in catchment models: 1. Evaluating parameter uncertainty , 1983 .

[4]  James C. Bennett,et al.  Accounting for seasonal dependence in hydrological model errors and prediction uncertainty , 2013 .

[5]  Andreas Scheidegger,et al.  Improving uncertainty estimation in urban hydrological modeling by statistically describing bias , 2013 .

[6]  Jery R. Stedinger,et al.  Condensed disaggregation procedures and conservation corrections for stochastic hydrology , 1988 .

[7]  Ashish Sharma,et al.  Hydrological model selection: A Bayesian alternative , 2005 .

[8]  D. H. Marks,et al.  A review and evaluation of multiobjective programing techniques , 1975 .

[9]  S. Sorooshian,et al.  Stochastic parameter estimation procedures for hydrologie rainfall‐runoff models: Correlated and heteroscedastic error cases , 1980 .

[10]  Soroosh Sorooshian,et al.  Model Parameter Estimation Experiment (MOPEX): An overview of science strategy and major results from the second and third workshops , 2006 .

[11]  G. Seber,et al.  Nonlinear Regression: Seber/Nonlinear Regression , 2005 .

[12]  J. Doherty,et al.  METHODOLOGIES FOR CALIBRATION AND PREDICTIVE ANALYSIS OF A WATERSHED MODEL 1 , 2003 .

[13]  M. Clark,et al.  Ancient numerical daemons of conceptual hydrological modeling: 2. Impact of time stepping schemes on model analysis and prediction , 2010 .

[14]  B. Bates,et al.  A Markov Chain Monte Carlo Scheme for parameter estimation and inference in conceptual rainfall‐runoff modeling , 2001 .

[15]  David Hinkley,et al.  On Quick Choice of Power Transformation , 1977 .

[16]  Chong-Yu Xu,et al.  Systematic evaluation of autoregressive error models as post-processors for a probabilistic streamflow forecast system , 2011 .

[17]  Julien Lerat,et al.  Crash testing hydrological models in contrasted climate conditions: An experiment on 216 Australian catchments , 2012 .

[18]  Bettina Schaefli,et al.  Quantifying hydrological modeling errors through a mixture of normal distributions , 2007 .

[19]  Quan J. Wang,et al.  Quantifying parameter uncertainty in stochastic models using the Box–Cox transformation , 2002 .

[20]  V. Singh,et al.  Mathematical Modeling of Watershed Hydrology , 2002 .

[21]  Anthony J. Jakeman,et al.  Performance of conceptual rainfall‐runoff models in low‐yielding ephemeral catchments , 1997 .

[22]  Q. J. Wang,et al.  Multisite probabilistic forecasting of seasonal flows for streams with zero value occurrences , 2011 .

[23]  Tyler Smith,et al.  Development of a formal likelihood function for improved Bayesian inference of ephemeral catchments , 2010 .

[24]  Christian Stamm,et al.  Integrated uncertainty assessment of discharge predictions with a statistical error model , 2013 .

[25]  D. Bauer.  Constructing Confidence Sets Using Rank Statistics , 1972 .

[26]  J. Marchal.  Cours d'economie politique , 1950 .

[27]  D. Cox,et al.  An Analysis of Transformations , 1964 .

[28]  Karl Pearson,et al.  Mathematical Contributions to the Theory of Evolution. XIX. Second Supplement to a Memoir on Skew Variation , 1901 .

[29]  Stefania Tamea,et al.  Verification tools for probabilistic forecasts of continuous hydrological variables , 2006 .

[30]  Dmitri Kavetski,et al.  Rainfall uncertainty in hydrological modelling: An evaluation of multiplicative error models , 2011 .

[31]  James C. Bennett,et al.  A System for Continuous Hydrological Ensemble Forecasting (SCHEF) to lead times of 9 days , 2014 .

[32]  P. R. Johnston,et al.  Parameter optimization for watershed models , 1976 .

[33]  T. McMahon,et al.  Updated world map of the Köppen-Geiger climate classification , 2007 .

[34]  George Kuczera,et al.  Toward a reliable decomposition of predictive uncertainty in hydrological modeling: Characterizing rainfall errors using conditional simulation , 2011 .

[35]  Dmitri Kavetski,et al.  Pursuing the method of multiple working hypotheses for hydrological modeling , 2011 .

[36]  George Kuczera,et al.  Critical evaluation of parameter consistency and predictive uncertainty in hydrological modeling: A case study using Bayesian total error analysis , 2009 .

[37]  P. Reichert,et al.  Hydrological modelling of the Chaohe Basin in China: Statistical model formulation and Bayesian inference , 2007 .

[38]  G. Villarini,et al.  Empirically-based modeling of spatial sampling uncertainties associated with rainfall measurements by rain gauges , 2008 .

[39]  Quan J. Wang,et al.  A log‐sinh transformation for data normalization and variance stabilization , 2012 .

[40]  David R. Cox,et al.  The Oxford Dictionary of Statistical Terms , 2006 .

[41]  T. C. Chamberlin.  The Method of Multiple Working Hypotheses: With this method the dangers of parental affection for a favorite theory can be circumvented. , 1965, Science.

[42]  Chong-yu Xu,et al.  Improvement and comparison of likelihood functions for model calibration and parameter uncertainty analysis within a Markov chain Monte Carlo scheme , 2014 .

[43]  Hoshin Vijai Gupta,et al.  Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling , 2009 .

[44]  R. Clarke.  A review of some mathematical models used in hydrology, with observations on their calibration and use , 1973 .

[45]  C. Perrin,et al.  Improvement of a parsimonious model for streamflow simulation , 2003 .

[46]  J. Stedinger,et al.  Generalized maximum‐likelihood generalized extreme‐value quantile estimators for hydrologic data , 2000 .

[47]  Q. Shao,et al.  Experimental evaluation of the dynamic seasonal streamflow forecasting approach , 2011 .

[48]  Francesca Pianosi,et al.  Dynamic modeling of predictive uncertainty by regression on absolute errors , 2012 .

[49]  George Kuczera,et al.  Comparison of joint versus postprocessor approaches for hydrological uncertainty estimation accounting for error autocorrelation and heteroscedasticity , 2014 .

[50]  John Doherty,et al.  Parameter estimation and uncertainty analysis for a watershed model , 2007, Environ. Model. Softw..

[51]  A. Raftery,et al.  Probabilistic forecasts, calibration and sharpness , 2007 .

[52]  Jin Teng,et al.  Climate non-stationarity – Validity of calibrated rainfall–runoff models for use in climate change studies , 2010 .

[53]  Tyler Smith,et al.  Modeling residual hydrologic errors with Bayesian inference , 2015 .

[54]  Peter C Young,et al.  Advances in real–time flood forecasting , 2002, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[55]  James C. Bennett,et al.  A strategy to overcome adverse effects of autoregressive updating of streamflow forecasts , 2015 .

[56]  Soroosh Sorooshian,et al.  Toward improved calibration of hydrologic models: Multiple and noncommensurable measures of information , 1998 .

[57]  Keith Beven,et al.  The future of distributed models: model calibration and uncertainty prediction. , 1992 .

[58]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[59]  George Kuczera,et al.  Pitfalls and improvements in the joint inference of heteroscedasticity and autocorrelation in hydrological model calibration , 2013 .

[60]  Henrik Madsen,et al.  An evaluation of the impact of model structure on hydrological modelling uncertainty for streamflow simulation , 2004 .

[61]  J. Vrugt,et al.  A formal likelihood function for parameter and predictive inference of hydrologic models with correlated, heteroscedastic, and non‐Gaussian errors , 2010 .