A framework to assess the realism of model structures using hydrological signatures

The use of flexible hydrological model structures for hypothesis testing requires an objective and diagnostic method to identify whether a rainfall-runoff model structure is suitable for a certain catchment. To determine if a model structure is realistic, i.e. if it captures the relevant runoff processes, both performance and consistency are important. We define performance as the ability of a model structure to mimic a specific part of the hydrological behaviour in a specific catchment. This can be assessed based on evaluation criteria, such as the goodness of fit of specific hydrological signatures obtained from hydrological data. Consistency is defined as the ability of a model structure to adequately reproduce several hydrological signatures simultaneously while using the same set of parameter values. In this paper we describe and demonstrate a new evaluation Framework for Assessing the Realism of Model structures (FARM). The evaluation framework tests for both performance and consistency using a principal component analysis on a range of evaluation criteria, all emphasizing different hydrological behaviour. The utility of this evaluation framework is demonstrated in a case study of two small headwater catchments (Maimai, New Zealand, and Wollefsbach, Luxembourg). Eight different hydrological signatures and eleven model structures have been used for this study. The results suggest that some model structures may reveal the same degree of performance for selected evaluation criteria while showing differences in consistency. The results also show that some model structures have a higher performance and consistency than others. The principal component analysis in combination with several hydrological signatures is shown to be useful to visualise the performance and consistency of a model structure for the study catchments. With this framework performance and consistency are evaluated to identify which model structure suits a catchment better compared to other model structures. Until now the framework has only been based on a qualitative analysis and not yet on a quantitative analysis.

[1]  Yuqiong Liu,et al.  Reconciling theory with observations: elements of a diagnostic approach to model evaluation , 2008 .

[2]  Thibault Mathevet,et al.  Hydrology and Earth System Sciences Hess Opinions " Crash Tests for a Standardized Evaluation of Hydrological Models " , 2022 .

[3]  J. Kirchner.  Getting the right answers for the right reasons: Linking measurements, analyses, and models to advance the science of hydrology , 2006 .

[4]  George Kuczera,et al.  Critical evaluation of parameter consistency and predictive uncertainty in hydrological modeling: A case study using Bayesian total error analysis , 2009 .

[5]  Hubert H. G. Savenije,et al.  On the calibration of hydrological models in ungauged basins: A framework for integrating hard and soft hydrological information , 2009 .

[6]  A. Brath,et al.  A stochastic approach for assessing the uncertainty of rainfall‐runoff simulations , 2004 .

[7]  Göran Lindström,et al.  Development and test of the distributed HBV-96 hydrological model , 1997 .

[8]  Wojtek J. Krzanowski,et al.  Principles of multivariate analysis : a user's perspective. oxford , 1988 .

[9]  Soroosh Sorooshian,et al.  Toward improved calibration of hydrologic models: Multiple and noncommensurable measures of information , 1998 .

[10]  W. Härdle,et al.  Applied Multivariate Statistical Analysis , 2003 .

[11]  Martyn P. Clark,et al.  Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models , 2008 .

[12]  Dmitri Kavetski,et al.  Elements of a flexible approach for conceptual hydrological modeling: 2. Application and experimental insights , 2011 .

[13]  Fred Worrall,et al.  Long-term changes in hydrological pathways in an upland peat catchment—recovery from severe drought? , 2006 .

[14]  Dmitri Kavetski,et al.  Pursuing the method of multiple working hypotheses for hydrological modeling , 2011 .

[15]  Doerthe Tetzlaff,et al.  Assessing the value of high‐resolution isotope tracer data in the stepwise development of a lumped conceptual rainfall–runoff model , 2010 .

[16]  Brian L. McGlynn,et al.  A review of the evolving perceptual model of hillslope flowpaths at the Maimai catchments, New Zealand , 2002 .

[17]  H. L. Penman.  Natural evaporation from open water, bare soil and grass , 1948, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[18]  Hoshin Vijai Gupta,et al.  Do Nash values have value? , 2007 .

[19]  Neil McIntyre,et al.  Towards reduced uncertainty in conceptual rainfall‐runoff modelling: dynamic identifiability analysis , 2003 .

[20]  Thorsten Wagener,et al.  Convergence of approaches toward reducing uncertainty in predictions in ungauged basins , 2011 .

[21]  Hoshin Vijai Gupta,et al.  Regionalization of constraints on expected watershed response behavior for improved predictions in ungauged basins , 2007 .

[22]  Keith Beven,et al.  Calibration of hydrological models using flow-duration curves , 2010 .

[23]  Hubert H. G. Savenije,et al.  Learning from model improvement: On the contribution of complementary data to process understanding , 2008 .

[24]  Richard A. Johnson,et al.  Applied Multivariate Statistical Analysis , 1983 .

[25]  Martyn P. Clark,et al.  Hydrological field data from a modeller's perspective: Part 1. Diagnostic tests for model structure , 2011 .

[26]  C. Perrin,et al.  Improvement of a parsimonious model for streamflow simulation , 2003 .

[27]  Hubert H. G. Savenije,et al.  A comparison of alternative multiobjective calibration strategies for hydrological modeling , 2007 .

[28]  J. Nash,et al.  River flow forecasting through conceptual models part I — A discussion of principles☆ , 1970 .

[29]  A. Bárdossy,et al.  Robust estimation of hydrological model parameters , 2008 .

[30]  B. Schaefli,et al.  Signature-based model calibration for hydrological prediction in mesoscale Alpine catchments , 2010 .

[31]  Keith Beven,et al.  Uniqueness of place and process representations in hydrological modelling , 2000 .

[32]  Xavier Descombes.  Stochastic Geometry for Image Analysis: Descombes/Stochastic Geometry for Image Analysis , 2013 .

[33]  Doerthe Tetzlaff,et al.  What can flux tracking teach us about water age distribution patterns and their temporal dynamics , 2012 .

[34]  Dmitri Kavetski,et al.  Elements of a flexible approach for conceptual hydrological modeling: 1. Motivation and theoretical development , 2011 .

[35]  P. Matgen,et al.  Understanding catchment behavior through stepwise model concept improvement , 2008 .

[36]  Hubert H. G. Savenije,et al.  HESS Opinions "The art of hydrology" , 2008 .

[37]  Dmitri Kavetski,et al.  Assessing the impact of mixing assumptions on the estimation of streamwater mean residence time , 2010 .

[38]  Soroosh Sorooshian,et al.  The role of hydrograph indices in parameter estimation of rainfall–runoff models , 2005 .

[39]  M. Sivapalan,et al.  Catchment classification: Empirical analysis of hydrologic similarity based on catchment function in the eastern USA , 2011 .

[40]  J. Seibert.  Multi-criteria calibration of a conceptual runoff model using a genetic algorithm , 2000 .

[41]  M. Sivapalan,et al.  Improving model structure and reducing parameter uncertainty in conceptual water balance models through the use of auxiliary data , 2005 .

[42]  Xavier Descombes,et al.  Stochastic geometry for image analysis , 2011 .

[43]  J. McDonnell,et al.  A decade of Predictions in Ungauged Basins (PUB)—a review , 2013 .

[44]  K. Beven,et al.  A limits of acceptability approach to model evaluation and uncertainty estimation in flood frequency estimation by continuous simulation: Skalka catchment, Czech Republic , 2009 .

[45]  Doerthe Tetzlaff,et al.  Conceptualization in catchment modelling: simply learning? , 2008 .

[46]  Hoshin Vijai Gupta,et al.  A process‐based diagnostic approach to model evaluation: Application to the NWS distributed hydrologic model , 2008 .

[47]  Jeffrey J. McDonnell,et al.  The role of event water, a rapid shallow flow component, and catchment size in summer stormflow , 1999 .

[48]  A. J. Pearce,et al.  Hydrology and related changes after harvesting native forest catchments and establishing pinus radiata plantations. Part 1. Introduction to study , 1994 .

[49]  Hubert H. G. Savenije,et al.  On the value of combined event runoff and tracer analysis to improve understanding of catchment functioning in a data-scarce semi-arid area , 2011 .

[50]  J. S. Verkade,et al.  Estimation of Predictive Hydrological Uncertainty using Quantile Regression , 2010 .

[51]  M. Sivapalan,et al.  Process controls of water balance variability in a large semi-arid catchment: downward approach to hydrological model development , 2001 .

[52]  I. Jolliffe.  Principal Component Analysis , 2002 .

[53]  Kellie B. Vaché,et al.  A process‐based rejectionist framework for evaluating catchment runoff model structure , 2006 .