Transferability of hydrological models and ensemble averaging methods between contrasting climatic periods

Understanding hydrological model predictive capabilities under contrasting climate conditions enables more robust decision making. Using Differential Split Sample Testing (DSST), we analyze the performance of six hydrological models for 37 Irish catchments under climate conditions unlike those used for model training. Additionally, we consider four ensemble averaging techniques when examining interperiod transferability. DSST is conducted using 2/3 year noncontinuous blocks of (i) the wettest/driest years on record based on precipitation totals and (ii) years with a more/less pronounced seasonal precipitation regime. Model transferability between contrasting regimes was found to vary depending on the testing scenario, catchment, and evaluation criteria considered. As expected, the ensemble average outperformed most individual ensemble members. However, averaging techniques differed considerably in the number of times they surpassed the best individual model member. Bayesian Model Averaging (BMA) and the Granger-Ramanathan Averaging (GRA) method were found to outperform the simple arithmetic mean (SAM) and Akaike Information Criteria Averaging (AICA). Here GRA performed better than the best individual model in 51%–86% of cases (according to the Nash-Sutcliffe criterion). When assessing model predictive skill under climate change conditions we recommend (i) setting up DSST to select the best available analogues of expected annual mean and seasonal climate conditions; (ii) applying multiple performance criteria; (iii) testing transferability using a diverse set of catchments; and (iv) using a multimodel ensemble in conjunction with an appropriate averaging technique. Given the computational efficiency and performance of GRA relative to BMA, the former is recommended as the preferred ensemble averaging technique for climate assessment.

[1]  Robert J. Abrahart,et al.  Multi-model data fusion for river flow forecasting: an evaluation of six alternative methods based on two contrasting catchments , 2002 .

[2]  Martyn P. Clark,et al.  Ensemble Bayesian model averaging using Markov Chain Monte Carlo sampling , 2008 .

[3]  Patrick M. Reed,et al.  Time‐varying sensitivity analysis clarifies the effects of watershed model formulation on model behavior , 2013 .

[4]  Martyn P. Clark,et al.  Diagnostic evaluation of multiple hypotheses of hydrological behaviour in a limits‐of‐acceptability framework for 24 UK catchments , 2014 .

[5]  Hoshin Vijai Gupta,et al.  Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling , 2009 .

[6]  J. Nash,et al.  River flow forecasting through conceptual models part I — A discussion of principles☆ , 1970 .

[7]  Martyn P. Clark,et al.  Comparison of hydrological model structures based on recession and low flow simulations , 2011 .

[8]  H. Akaike.  A new look at the statistical model identification , 1974 .

[9]  K. Burnham,et al.  Model selection: An integral part of inference , 1997 .

[10]  F. Anctil,et al.  Improving the temporal transposability of lumped hydrological models on twenty diversified U.S. watersheds , 2015 .

[11]  Jamie Hannaford,et al.  High‐flow and flood trends in a network of undisturbed catchments in the UK , 2008 .

[12]  F. Anctil,et al.  Can a multi-model approach improve hydrological ensemble forecasting? A study on 29 French catchments using 16 hydrological model structures , 2011 .

[13]  S. Steinschneider,et al.  A climate change range‐based method for estimating robustness for water resources supply , 2014 .

[14]  Bruce E. Hansen,et al.  Least-squares forecast averaging , 2008 .

[15]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[16]  P. Krause,et al.  COMPARISON OF DIFFERENT EFFICIENCY CRITERIA FOR HYDROLOGICAL MODEL ASSESSMENT , 2005 .

[17]  Bryan A. Tolson,et al.  Optimizing hydrological consistency by incorporating hydrological signatures into model calibration objectives , 2015 .

[18]  A. Bárdossy,et al.  Investigation of the transferability of hydrological models and a method to improve model calibration , 2005 .

[19]  C. Prudhomme,et al.  Low Flow Response Surfaces for Drought Decision Support: A Case Study from the UK , 2015 .

[20]  Ludovic Oudin,et al.  Hydrological model parameter instability: A source of additional uncertainty in estimating the hydrological impacts of climate change? , 2013 .

[21]  C. Perrin,et al.  Improvement of a parsimonious model for streamflow simulation , 2003 .

[22]  M. Clyde,et al.  Bayesian model averaging: A tutorial - Comment , 1999 .

[23]  S. Bastola,et al.  Generating probabilistic estimates of hydrological response for Irish catchments using a weather generator and probabilistic climate change scenarios , 2012 .

[24]  Thibault Mathevet,et al.  Hydrology and Earth System Sciences Hess Opinions " Crash Tests for a Standardized Evaluation of Hydrological Models " , 2022 .

[25]  S. Steele‐Dunne,et al.  The impacts of climate change on hydrology in Ireland , 2008 .

[26]  A. Raftery,et al.  Using Bayesian Model Averaging to Calibrate Forecast Ensembles , 2005 .

[27]  Martin F. Lambert,et al.  A strategy for diagnosing and interpreting hydrological model nonstationarity , 2014 .

[28]  H. Price.  Past and future , 1990, Nature.

[29]  Walter Boughton,et al.  The Australian water balance model , 2004, Environ. Model. Softw..

[30]  Hubert H. G. Savenije,et al.  A framework to assess the realism of model structures using hydrological signatures , 2012 .

[31]  Dmitri Kavetski,et al.  Catchment properties, function, and conceptual model representation: is there a correspondence? , 2014 .

[32]  D. Wheeler,et al.  Regional weather and climates of the British Isles ‐ Part 1: Introduction , 2013 .

[33]  François Anctil,et al.  Performance and reliability of multimodel hydrological ensemble simulations based on seventeen lumped models and a thousand catchments , 2010 .

[34]  Thibault Mathevet,et al.  A downward structural sensitivity analysis of hydrological models to improve low-flow simulation , 2011 .

[35]  R. Wilby,et al.  An evaluation of persistent meteorological drought using a homogeneous Island of Ireland precipitation network , 2016 .

[36]  Julien Lerat,et al.  Crash testing hydrological models in contrasted climate conditions: An experiment on 216 Australian catchments , 2012 .

[37]  Richard J. Beckman,et al.  A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code , 2000, Technometrics.

[38]  J. Sweeney.  Regional weather and climates of the British Isles – Part 6: Ireland , 2014 .

[39]  Martyn P. Clark,et al.  Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models , 2008 .

[40]  Tim Hess,et al.  Which catchment characteristics control the temporal dependence structure of daily river flows? , 2014 .

[41]  Tiesong Hu,et al.  River flow time series prediction with a range-dependent neural network , 2001 .

[42]  Martyn P. Clark,et al.  Characterizing Uncertainty of the Hydrologic Impacts of Climate Change , 2016, Current Climate Change Reports.

[43]  M. D. McKay,et al.  A comparison of three methods for selecting values of input variables in the analysis of output from a computer code , 2000 .

[44]  Thibault Mathevet,et al.  Hydrology under change: an evaluation protocol to investigate how hydrological models deal with changing catchments , 2015 .

[45]  Dmitri Kavetski,et al.  Elements of a flexible approach for conceptual hydrological modeling: 1. Motivation and theoretical development , 2011 .

[46]  Keith Beven,et al.  Multi-period and multi-criteria model conditioning to reduce prediction uncertainty in an application of TOPMODEL within the GLUE framework , 2007 .

[47]  Adrian E. Raftery,et al.  Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors , 1999 .

[48]  Michael D. McKay,et al.  The prediction uncertainty , 2003 .

[49]  François Anctil,et al.  Impact of the length of observed records on the performance of ANN and of conceptual parsimonious rainfall-runoff forecasting models , 2004, Environ. Model. Softw..

[50]  Rebecca Campbell,et al.  A Comparison of Three Methods , 2003 .

[51]  C. Perrin,et al.  Does a large number of parameters enhance model performance? Comparative assessment of common catchment model structures on 429 catchments , 2001 .

[52]  Shane Dawson,et al.  Loughborough University Institutional Repository The Statistical Downscaling Model-Decision Centric ( SDSM-DC ) : conceptual basis and applications , 2014 .

[53]  Vazken Andréassian,et al.  Spatial proximity, physical similarity, regression and ungaged catchments: A comparison of regionalization approaches based on 913 French catchments , 2008 .

[54]  Stan Openshaw,et al.  A hybrid multi-model approach to river level forecasting , 2000 .

[55]  P. E. O'connell,et al.  River flow forecasting through conceptual models part III - The Ray catchment at Grendon Underwood , 1970 .

[56]  S. Uhlenbrook,et al.  Prediction uncertainty of conceptual rainfall-runoff models caused by problems in identifying model parameters and structure , 1999 .

[57]  Neil McIntyre,et al.  Towards reduced uncertainty in conceptual rainfall‐runoff modelling: dynamic identifiability analysis , 2003 .

[58]  Soroosh Sorooshian,et al.  A framework for development and application of hydrological models , 2001, Hydrology and Earth System Sciences.

[59]  Keith Beven,et al.  The future of distributed models: model calibration and uncertainty prediction. , 1992 .

[60]  R. Wilby,et al.  Persistence of Hydrometeorological Droughts in the United Kingdom: A Regional Analysis of Multi-Season Rainfall and River Flow Anomalies , 2015 .

[61]  Guillaume Thirel,et al.  On the need to test hydrological models under changing conditions , 2015 .

[62]  Günter Blöschl,et al.  Time stability of catchment model parameters: Implications for climate impact analyses , 2011 .

[63]  John P. Bloomfield,et al.  Examining geological controls on baseflow index (BFI) using regression analysis: An illustration from the Thames Basin, UK , 2009 .

[64]  L. S. Pereira,et al.  Crop evapotranspiration : guidelines for computing crop water requirements , 1998 .

[65]  Murugesu Sivapalan,et al.  Comparative assessment of predictions in ungauged basins – Part 1: Runoff-hydrograph studies , 2013 .

[66]  V. Klemeš,et al.  Operational Testing of Hydrological Simulation Models , 2022 .

[67]  Thorsten Wagener.  Evaluation of catchment models , 2003 .

[68]  D. Marchant,et al.  The diagnostic evaluation. , 2002, Obstetrics and gynecology clinics of North America.

[69]  F. Brissette,et al.  A comparative analysis of 9 multi-model averaging approaches in hydrological continuous streamflow simulation , 2015 .

[70]  R. Wilby,et al.  Scenario-neutral approach to climate change impact studies: Application to flood risk , 2010 .

[71]  François Anctil,et al.  Multimodel evaluation of twenty lumped hydrological models under contrasted climate conditions , 2011 .

[72]  R. Woods,et al.  Climate and landscape controls on water balance model complexity over changing timescales , 2002 .

[73]  Dmitri Kavetski,et al.  The influence of conceptual model structure on model performance: a comparative study for 237 French catchments , 2013 .

[74]  S. Walsh.  01 - NEW LONG-TERM RAINFALL AVERAGES FOR IRELAND , 2012 .

[75]  Alain Dassargues,et al.  Conceptual model uncertainty in groundwater modeling: Combining generalized likelihood uncertainty estimation and Bayesian model averaging , 2008 .

[76]  R. Wilby,et al.  A framework for assessing uncertainties in climate change impacts: Low‐flow scenarios for the River Thames, UK , 2006 .

[77]  S. Sorooshian,et al.  Automatic calibration of conceptual rainfall-runoff models: sensitivity to calibration data , 1996 .

[78]  A. Shamseldin,et al.  Methods for combining the outputs of different rainfall–runoff models , 1997 .

[79]  Roger E. Smith,et al.  Mathematical simulation of interdependent surface and subsurface hydrologic processes , 1983 .

[80]  Lu Zhang,et al.  The transferability of hydrological models under nonstationary climatic conditions , 2011 .

[81]  R. Wilby,et al.  Past and future climate change in the context of memorable seasonal extremes , 2016 .

[82]  C. Jackson,et al.  Hillslope infiltration and lateral downslope unsaturated flow , 1992 .

[83]  AUTOMATIC CALIBRATION OF A CONCEPTUAL RAINFALL-RUNOFF MODEL USING SIMULTANEOUSLY SEVERAL HYDROLOGICAL EVENTS , 2010 .

[84]  Henrik Madsen,et al.  Automatic calibration of a conceptual rainfall-runoff model using multiple objectives. , 2000 .

[85]  Robert L. Wilby,et al.  Uncertainty in water resource model parameters used for climate change impact assessment , 2005 .

[86]  Thibault Mathevet,et al.  Impact of biased and randomly corrupted inputs on the efficiency and the parameters of watershed models , 2006 .

[87]  S. Sorooshian,et al.  Multimodel Combination Techniques for Analysis of Hydrological Simulations: Application to Distributed Model Intercomparison Project Results , 2006 .

[88]  S. Bastola,et al.  The role of hydrological modelling uncertainties in climate change impact assessments of Irish river catchments , 2011 .

[89]  K. Lam,et al.  River flow time series prediction with a range-dependent neural network , 2001 .

[90]  Thibault Mathevet,et al.  Dynamic averaging of rainfall‐runoff model simulations from complementary model parameterizations , 2006 .

[91]  Henrik Madsen,et al.  Including prior information in the estimation of effective soil parameters in unsaturated zone modelling , 2004 .

[92]  Jasper A. Vrugt,et al.  Comparison of point forecast accuracy of model averaging methods in hydrologic applications , 2010 .

[93]  F. H. Dawson,et al.  Groundwater dominated rivers , 1999 .

[94]  Jin Teng,et al.  Climate non-stationarity – Validity of calibrated rainfall–runoff models for use in climate change studies , 2010 .

[95]  Keith Beven,et al.  A manifesto for the equifinality thesis , 2006 .

[96]  K. Price.  Effects of watershed topography, soils, land use, and climate on baseflow hydrology in humid regions: A review , 2011 .

[97]  P. Coulibaly,et al.  Identification of rainfall–runoff model for improved baseflow estimation in ungauged basins , 2012 .

[98]  Henrik Madsen,et al.  Generalized likelihood uncertainty estimation (GLUE) using adaptive Markov Chain Monte Carlo sampling , 2008 .

[99]  François Anctil,et al.  Accounting for three sources of uncertainty in ensemble hydrological forecasting , 2015 .

[100]  Conor Murphy,et al.  Climate-driven trends in mean and high flows from a network of reference stations in Ireland , 2013 .