On hypothesis testing in hydrology: Why falsification of models is still a really good idea

This opinion piece argues that in respect of testing models as hypotheses about how catchments function, there is no existing methodology that adequately deals with the potential for epistemic uncertainties about data and hydrological processes in the modelling processes. A rejectionist framework is suggested as a way ahead, wherein assessments of uncertainties in the input and evaluation data are used to define limits of acceptability prior to any model simulations being made. The limits of acceptability might also depend on the purpose of the modelling so that we can be more rigorous about whether a model is actually fit-for-purpose. Different model structures and parameter sets can be evaluated in this framework, albeit that subjective elements necessarily remain, given the epistemic nature of the uncertainties in the modelling process. One of the most effective ways of reducing the impacts of epistemic uncertainties, and allow more rigorous hypothesis testing, would be to commission better observational methods. Model rejection is a good thing in that it requires us to be better, resulting in advancement of the science.

[1]  Günter Blöschl,et al.  Debates—Hypothesis testing in hydrology: Introduction , 2017 .

[2]  K. Beven,et al.  A limits of acceptability approach to model evaluation and uncertainty estimation in flood frequency estimation by continuous simulation: Skalka catchment, Czech Republic , 2009 .

[3]  Andrew Binley,et al.  GLUE: 20 years on , 2014 .

[4]  Keith Beven,et al.  Dalton Medal Lecture: How far can we go in distributed hydrological modelling? , 2001 .

[5]  Hoshin Vijai Gupta,et al.  The quantity and quality of information in hydrologic models , 2015 .

[6]  Keith Beven,et al.  A discrete particle representation of hillslope hydrology: hypothesis testing in reproducing a tracer experiment at Gårdsjön, Sweden , 2011 .

[7]  Keith Beven,et al.  Multi-period and multi-criteria model conditioning to reduce prediction uncertainty in an application of TOPMODEL within the GLUE framework , 2007 .

[8]  Karel J. Keesman,et al.  Set Membership Approach to Identification and Prediction of Lake Eutrophication , 1990 .

[9]  J. Kirchner.  Getting the right answers for the right reasons: Linking measurements, analyses, and models to advance the science of hydrology , 2006 .

[10]  Keith Beven,et al.  Multiple sources of predictive uncertainty in modeled estimates of net ecosystem CO2 exchange , 2009 .

[11]  Keith Beven,et al.  Processes influencing model-data mismatch in drought-stressed, fire-disturbed eddy flux sites , 2010 .

[12]  Keith Beven,et al.  Causal models as multiple working hypotheses about environmental processes , 2012 .

[13]  Keith Beven,et al.  The challenges of modelling phosphorus in a headwater catchment: Applying a ‘limits of acceptability’ uncertainty framework to a water quality model , 2018 .

[14]  M. Kirkby.  Tests of the random network model, and its application to basin hydrology , 1976 .

[15]  Victor R. Baker,et al.  Debates—Hypothesis testing in hydrology: Pursuing certainty versus pursuing uberty , 2017 .

[16]  Yeosang Yoon,et al.  Estimating river bathymetry from data assimilation of synthetic SWOT measurements , 2012 .

[17]  Laurent Pfister,et al.  Debates—Hypothesis testing in hydrology: Theory and practice , 2017 .

[18]  Alfred O. Hero,et al.  Estimating epistemic and aleatory uncertainties during hydrologic modeling: An information theoretic approach , 2013 .

[19]  Diane M. McKnight.  Debates—Hypothesis testing in hydrology: A view from the field: The value of hydrologic hypotheses in designing field studies and interpreting the results to advance hydrology , 2017 .

[20]  Keith Beven,et al.  Facets of uncertainty: epistemic uncertainty, non-stationarity, likelihood, hypothesis testing, and communication , 2016 .

[21]  Keith Beven,et al.  Framework for assessing uncertainty in fluvial flood risk mapping , 2014 .

[22]  Keith Beven,et al.  Modelling the Chloride Signal at the Plynlimon Catchments, Wales Using a Modified Dynamic TOPMODEL. , 2007 .

[23]  Keith Beven,et al.  Advice to a young hydrologist , 2016 .

[24]  V. Klemeš,et al.  Operational Testing of Hydrological Simulation Models , 2022 .

[25]  Rainer Helmig,et al.  Debates - Hypothesis testing in hydrology: A subsurface perspective , 2017 .

[26]  Dmitri Kavetski,et al.  A unified approach for process‐based hydrologic modeling: 2. Model implementation and case studies , 2015 .

[27]  Lotfi A. Zadeh,et al.  Generalized theory of uncertainty (GTU) - principal concepts and ideas , 2006, Comput. Stat. Data Anal..

[28]  Keith Beven,et al.  Modelling the chloride signal at Plynlimon, Wales, using a modified dynamic TOPMODEL incorporating conservative chemical mixing (with uncertainty) , 2007 .

[29]  Kenneth A. Rose,et al.  Parameter sensitivities, monte carlo filtering, and model forecasting under uncertainty , 1991 .

[30]  Keith Beven,et al.  Uncertainty assessment of a process-based integrated catchment model of phosphorus , 2009 .

[31]  Lotfi A. Zadeh,et al.  Toward a generalized theory of uncertainty (GTU) - an outline , 2005, GrC.

[32]  P. Dirmeyer,et al.  The plumbing of land surface models: is poor performance a result of methodology or data quality? , 2016, Journal of hydrometeorology.

[33]  Keith Beven,et al.  Comment on “Pursuing the method of multiple working hypotheses for hydrological modeling” by P. Clark et al. , 2012 .

[34]  Jim Freer,et al.  Towards a limits of acceptability approach to the calibration of hydrological models : Extending observation error , 2009 .

[35]  K. Beven,et al.  The hydrological response of headwater and sideslope areas / La réponse hydrologique des zones de cours supérieurs et des zones de pente latérale , 1978 .

[36]  K. Beven,et al.  Integrated modeling of flow and residence times at the catchment scale with multiple interacting pathways , 2013 .

[37]  Keith Beven,et al.  Uniqueness of place and process representations in hydrological modelling , 2000 .

[38]  Martyn P. Clark,et al.  Benchmarking of a Physically Based Hydrologic Model , 2017 .

[39]  Keith Beven,et al.  On doing better hydrological science , 2008 .

[40]  Karel J. Keesman,et al.  Uncertainty propagation and speculation in projective forecasts of environmental change - a lake eutrophication example. , 1991 .

[41]  P. Dirmeyer,et al.  The Plumbing of Land Surface Models: Benchmarking Model Performance , 2015 .

[42]  Keith Beven,et al.  On virtual observatories and modelled realities (or why discharge must be treated as a virtual variable) , 2012 .

[43]  Ataur Rahman,et al.  Selection of the best fit flood frequency distribution and parameter estimation procedure: a case study for Tasmania in Australia , 2011 .

[44]  Edward Beltrami,et al.  Uncertainty and Information , 2020, What Is Random?.

[45]  Keith Beven,et al.  Concepts of Information Content and Likelihood in Parameter Calibration for Hydrological Simulation Models , 2014 .

[46]  George J. Klir,et al.  Uncertainty and Information: Emergence of Vast New Territories , 2006 .

[47]  Dmitri Kavetski,et al.  Pursuing the method of multiple working hypotheses for hydrological modeling , 2011 .

[48]  Keith Beven,et al.  A manifesto for the equifinality thesis , 2006 .

[49]  Richard M. Vogel,et al.  Hypothesis tests for hydrologic alteration , 2015 .

[50]  M. Clark,et al.  A philosophical basis for hydrological uncertainty , 2016 .

[51]  Sujay V. Kumar,et al.  Benchmarking NLDAS-2 Soil Moisture and Evapotranspiration to Separate Uncertainty Contributions. , 2016, Journal of hydrometeorology.