A Recommended Calibration and Validation Strategy for Hydrologic and Water Quality Models

Abstract. Hydrologic and water quality (H/WQ) models are widely used to support site-specific environmental assessment, design, planning, and decision making. Calibration and validation (C/V) are fundamental processes used to demonstrate that an H/WQ model can produce suitable results in a particular application. However, the lack of comprehensive guidelines has led to the use of ad hoc, inconsistent, and incomplete C/V processes, which have made it difficult to interpret the myriad of published modeling studies, reduced the utility of many modeling applications, and slowed the advancement of H/WQ modeling. The objective of this article is to provide a generalized structure and process to assist modelers in developing a C/V strategy for H/WQ modeling applications. These best practice recommendations were developed based on an expansive review of the modeling literature, including a special collection of articles on H/WQ model calibration, validation, and use, as well as extensive discussion and debate among the authors. The model C/V recommendations include careful consideration, execution, and documentation of the following elements: (1) goals of model use, (2) data and parameters used in C/V, and (3) model C/V processes. Considerations in element 3 include the warm-up period, C/V strategy complexity, C/V process staging, spatiotemporal allocation of C/V comparison data, manual vs. automatic C/V, and additional diagnostics. Notable examples from the literature are provided for each strategy element. The comprehensive C/V strategy described herein will allow for better interpretation of future modeling studies, improved utility of modeling applications, and more systematic advancement of H/WQ models.

[1]  A. B. Bottcher,et al.  Watershed Assessment Model (WAM): Model Use, Calibration, and Validation , 2012 .

[2]  W. G. Knisel,et al.  CREAMS/GLEAMS: model use, calibration, and validation. , 2012 .

[3]  Henrik Madsen,et al.  Parameter estimation in distributed hydrological modelling: comparison of global and local optimisation techniques , 2007 .

[4]  Kyle R. Douglas-Mankin,et al.  Soil and Water Assessment Tool (SWAT) Hydrologic/Water Quality Model: Extended Capability and Wider Adoption , 2011 .

[5]  John D. Bredehoeft,et al.  Ground-water models cannot be validated , 1992 .

[6]  John Doherty,et al.  Comparison of hydrologic calibration of HSPF using automatic and manual methods , 2007 .

[7]  Sanjay Shukla,et al.  MIKE SHE: model use, calibration, and validation. , 2012 .

[8]  James V. Bonta,et al.  Hydrologic and water quality modeling: Spatial and temporal considerations , 2015 .

[9]  S. Sorooshian,et al.  Effective and efficient global optimization for conceptual rainfall‐runoff models , 1992 .

[10]  Naresh Pai,et al.  Identifying Priority Subwatersheds in the Illinois River Drainage Area in Arkansas Watershed Using a Distributed Modeling Approach , 2011 .

[11]  Kyle R. Mankin,et al.  Comparison of AnnAGNPS and SWAT model simulation results in USDA‐CEAP agricultural watersheds in south‐central Kansas , 2009 .

[12]  Sean A. Woznicki,et al.  ASSESSING BEST MANAGEMENT PRACTICE IMPLEMENTATION STRATEGIES UNDER CLIMATE CHANGE SCENARIOS , 2011 .

[13]  S. E. Rantz,et al.  Measurement and computation of streamflow: Volume 1, Measurement of stage and discharge , 1982 .

[14]  Shiying Tian,et al.  DRAINMOD-FOREST: Integrated Modeling of Hydrology, Soil Carbon and Nitrogen Dynamics, and Plant Growth for Drained Forests. , 2012, Journal of environmental quality.

[15]  Kyle R. Douglas-Mankin,et al.  Hydrologic and Water Quality Models: Use, Calibration, and Validation , 2012 .

[16]  Stefan Finsterle,et al.  Hydrologic and Water Quality Models: Sensitivity , 2015 .

[17]  Kyle R. Douglas-Mankin,et al.  Seasonal and Annual Impacts of Climate Change on Watershed Response Using an Ensemble of Global Climate Models , 2011 .

[18]  Kati W. Migliaccio,et al.  Modifying Goodness-of-Fit Indicators to Incorporate Both Measurement and Model Uncertainty in Model Calibration and Validation , 2010 .

[19]  Hoshin V. Gupta,et al.  On the use of spatial regularization strategies to improve calibration of distributed watershed models , 2010 .

[20]  Timothy A. Cohn,et al.  Load Estimator (LOADEST): A FORTRAN Program for Estimating Constituent Loads in Streams and Rivers , 2004 .

[21]  R. S. Dinicola Rainfall-Runoff Relations for the Puget Sound Area , 1992 .

[22]  Kyle R. Douglas-Mankin,et al.  Field-Level Targeting Using SWAT: Mapping Output from HRUs to Fields and Assessing Limitations of GIS Input Data , 2011 .

[23]  I BOEREMA,et al.  [A synthesis]. , 1960, Nederlands tijdschrift voor geneeskunde.

[24]  James C. Ascough,et al.  WEPP: Model Use, Calibration, and Validation , 2012 .

[25]  D. R. Smith,et al.  Estimating storm discharge and water quality data uncertainty: A software tool for monitoring and modeling applications , 2009, Environ. Model. Softw..

[26]  J. Hernandez,et al.  Uncertainty Considerations in Calibration and Validation of Hydrologic and Water Quality Models , 2015 .

[27]  J. Refsgaard Parameterisation, calibration and validation of distributed hydrological models , 1997 .

[28]  C. W. Chen,et al.  WARMF: Model Use, Calibration, and Validation , 2012 .

[29]  Liwang Ma,et al.  Root Zone Water Quality Model (RZWQM2): Model Use, Calibration, and Validation , 2012 .

[30]  Jane R. Frankenberger,et al.  Hydrologic and Water Quality Terminology as Applied to Modeling , 2015 .

[31]  James V. Bonta,et al.  Parameterization Guidelines and Considerations for Hydrologic Models , 2015 .

[32]  Xuesong Zhang,et al.  SWAT Ungauged: Hydrological Budget and Crop Yield Predictions in the Upper Mississippi River Basin , 2010 .

[33]  J. Arnold,et al.  VALIDATION OF THE SWAT MODEL ON A LARGE RWER BASIN WITH POINT AND NONPOINT SOURCES 1 , 2001 .

[34]  Anthony S. Donigian,et al.  BASINS/HSPF: MODEL USE, CALIBRATION, AND VALIDATION , 2012 .

[35]  K. R. Douglas-Mankin,et al.  Evaluating, interpreting, and communicating performance of hydrologic/water quality models considering intended use: A review and recommendations , 2014, Environ. Model. Softw..

[36]  Naresh Pai,et al.  Hydrologic and Water Quality Models: Performance Measures and Evaluation Criteria , 2015 .

[37]  Youssef,et al.  Hydrological Processes and Model Representation: Impact of Soft Data on Calibration , 2015 .

[38]  Jaehak Jeong,et al.  EPIC and APEX: Model Use, Calibration, and Validation , 2012 .

[39]  D. K. Borah,et al.  SEDIMENT AND NUTRIENT MODELING FOR TMDL DEVELOPMENT AND IMPLEMENTATION , 2006 .

[40]  Raghavan Srinivasan,et al.  Assessing the Implications of Water Harvesting Intensification on Upstream-downstream Social-ecological systems : a Case Study in the Lake Tana Basin , 2014 .

[41]  H. Gao,et al.  USE OF A GENETIC ALGORITHM AND MULTI-OBJECTIVE PROGRAMMING FOR CALIBRATION OF A HYDROLOGIC MODEL , 1998 .

[42]  Anthony J. Jakeman,et al.  Ten iterative steps in development and evaluation of environmental models , 2006, Environ. Model. Softw..

[43]  David D. Bosch,et al.  PROBLEMS AND POTENTIAL OF AUTOCALIBRATING A HYDROLOGIC MODEL , 2005 .

[44]  R. Daren Harmel,et al.  Consideration of measurement uncertainty in the evaluation of goodness-of-fit in hydrologic and water quality modeling , 2007 .

[45]  R. Srinivasan,et al.  High-Resolution Simulations of Decadal Climate Variability Impacts on Water Yield in the Missouri River Basin with the Soil and Water Assessment Tool (SWAT) , 2016 .

[46]  Bernard A. Engel,et al.  A mixed discrete‐continuous variable multiobjective genetic algorithm for targeted implementation of nonpoint source pollution control practices , 2013 .

[47]  James W. Jones,et al.  DRAINMOD-DSSAT model for simulating hydrology, soil carbon and nitrogen dynamics, and crop growth for drained crop land , 2014 .

[48]  Jörg Dietrich,et al.  Evaluation of different calibration strategies for large scale continuous hydrological modelling , 2012 .

[49]  James C. Ascough,et al.  ROOT ZONE WATER QUALITY MODEL SENSITIVITY ANALYSIS USING MONTE CARLO SIMULATION , 2000 .

[50]  Soroosh Sorooshian,et al.  Optimal use of the SCE-UA global optimization method for calibrating watershed models , 1994 .

[51]  Jirka Simunek,et al.  HYDRUS: Model Use, Calibration, and Validation , 2012 .

[52]  R. P. Silberstein,et al.  Hydrological models are so good, do we still need data? , 2006, Environ. Model. Softw..

[53]  Joseph H. A. Guillaume,et al.  Characterising performance of environmental models , 2013, Environ. Model. Softw..

[54]  Prasanna H. Gowda,et al.  ADAPT: Model Use, Calibration, and Validation , 2012 .

[55]  Raghavan Srinivasan,et al.  Spatio-Temporal Impacts of Biofuel Production and Climate Variability on Water Quantity and Quality in Upper Mississippi River Basin , 2015 .

[56]  R Wayne Skaggs,et al.  Field evaluation of a model for predicting nitrogen losses from drained lands. , 2006, Journal of environmental quality.

[57]  Jens Christian Refsgaard,et al.  Towards a Formal Approach to Calibration and Validation of Models Using Spatial Data , 2004 .

[58]  R. W. Skaggs,et al.  SENSITIVITY ANALYSES OF THE NITROGEN SIMULATION MODEL, DRAINMOD-N II , 2005 .

[59]  Soroosh Sorooshian,et al.  Status of Automatic Calibration for Hydrologic Models: Comparison with Multilevel Expert Calibration , 1999 .

[60]  Jens Christian Refsgaard,et al.  Construction, Calibration And Validation of Hydrological Models , 1990 .

[61]  Raghavan Srinivasan,et al.  SWAT: Model Use, Calibration, and Validation , 2012 .

[62]  Soroosh Sorooshian,et al.  Toward improved streamflow forecasts: value of semidistributed modeling , 2001 .

[63]  Ambro Gieske,et al.  Calibration of a semi-distributed hydrological model using discharge and remote sensing data , 2009 .

[64]  Ge Sun,et al.  Streamflow Response to Climate and Landuse Changes in a Coastal Watershed in North Carolina , 2009 .

[65]  D. Goodrich,et al.  KINEROS2/AGWA: Model use, calibration and validation , 2012 .

[66]  Jeffrey G. Arnold,et al.  CUMULATIVE UNCERTAINTY IN MEASURED STREAMFLOW AND WATER QUALITY DATA FOR SMALL WATERSHEDS , 2006 .

[67]  Søren Hansen,et al.  Daisy: Model Use, Calibration, and Validation , 2012 .

[68]  Devendra M. Amatya,et al.  Global sensitivity analysis of DRAINMOD‐FOREST, an integrated forest ecosystem model , 2014 .

[69]  John Doherty,et al.  REPLY TO DISCUSSION by Ray C. Whittemore.: "Methodologies for Calibration and Predictive Analysis of a Watershed Model" , 2004 .

[70]  R. W. Skaggs,et al.  DRAINMOD: Model Use, Calibration, and Validation , 2012 .

[71]  David Huard,et al.  Calibration of hydrological model GR2M using Bayesian uncertainty analysis , 2008 .

[72]  J. Arnold,et al.  Suitability of SWAT for the Conservation Effects Assessment Project: Comparison on USDA Agricultural Research Service Watersheds , 2007 .

[73]  Ayalew Kassahun,et al.  Quality assurance in model based water management - review of existing practice and outline of new approaches , 2005, Environ. Model. Softw..

[74]  Prem B. Parajuli,et al.  Assessing sensitivity of hydrologic responses to climate change from forested watershed in Mississippi , 2010 .

[75]  Kyle R. Douglas-Mankin,et al.  SWAT Discharge Response to Spatial Rainfall Variability in a Kansas Watershed , 2010 .

[76]  Mazdak Arabi,et al.  Impact of climate change on diffuse pollutant fluxes at the watershed scale , 2014 .

[77]  Kyle R. Douglas-Mankin,et al.  NEXRAD Stage III Precipitation Local Bias Adjustment for Streamflow Prediction , 2010 .

[78]  Indrajeet Chaubey,et al.  Uncertainty in TMDL Models , 2006 .

[79]  Raghavan Srinivasan,et al.  Evaluation of global optimization algorithms for parameter calibration of a computationally intensive hydrologic model , 2009 .

[80]  Mazdak Arabi,et al.  Global sensitivity and uncertainty analysis of a dynamic agroecosystem model under different irrigation treatments , 2012 .

[81]  Stuart P. Hardegree,et al.  Simultaneous heat and water (SHAW) model: model use, calibration, and validation. , 2012 .

[82]  Kyle R. Mankin,et al.  Calibration and Validation of ADAPT and SWAT for Field‐Scale Runoff Prediction 1 , 2007 .

[83]  Bryan A. Tolson,et al.  An efficient framework for hydrologic model calibration on long data periods , 2013 .

[84]  Indrajeet Chaubey,et al.  SENSITIVITY ANALYSIS, CALIBRATION, AND VALIDATIONS FOR A MULTISITE AND MULTIVARIABLE SWAT MODEL 1 , 2005 .

[85]  N. Crawford,et al.  The synthesis of continuous streamflow hydrographs on a digital computer , 1962 .

[86]  Jeffrey G. Arnold,et al.  Model Evaluation Guidelines for Systematic Quantification of Accuracy in Watershed Simulations , 2007 .

[87]  Sabine Grunwald,et al.  GIS-BASED HYDROLOGIC MODELING IN THE SANDUSKY WATERSHED USING SWAT , 2005 .