Towards a comprehensive assessment of model structural adequacy

[1] The past decade has seen significant progress in characterizing uncertainty in environmental systems models, through statistical treatment of incomplete knowledge regarding parameters, model structure, and observational data. Attention has now turned to the issue of model structural adequacy (MSA, a term we prefer over model structure “error”). In reviewing philosophical perspectives from the groundwater, unsaturated zone, terrestrial hydrometeorology, and surface water communities about how to model the terrestrial hydrosphere, we identify several areas where different subcommunities can learn from each other. In this paper, we (a) propose a consistent and systematic “unifying conceptual framework” consisting of five formal steps for comprehensive assessment of MSA; (b) discuss the need for a pluralistic definition of adequacy; (c) investigate how MSA has been addressed in the literature; and (d) identify four important issues that require detailed attention—structured model evaluation, diagnosis of epistemic cause, attention to appropriate model complexity, and a multihypothesis approach to inference. We believe that there exists tremendous scope to collectively improve the scientific fidelity of our models and that the proposed framework can help to overcome barriers to communication. By doing so, we can make better progress toward addressing the question “How can we use data to detect, characterize, and resolve model structural inadequacies?”

[1]  T Prabhakar Clement,et al.  Complexities in Hindcasting Models—When Should We Say Enough Is Enough? , 2011, Ground water.

[2]  M. Ek,et al.  Hyperresolution global land surface modeling: Meeting a grand challenge for monitoring Earth's terrestrial water , 2011 .

[3]  Murugesu Sivapalan,et al.  Power law catchment‐scale recessions arising from heterogeneous linear small‐scale dynamics , 2009 .

[4]  J. A. Vrugt,et al.  Bayesian Selection of Hydrological Models Using Sequential Monte Carlo Sampling , 2011 .

[5]  H. Gupta,et al.  Real-Time Data Assimilation for Operational Ensemble Streamflow Forecasting , 2006 .

[6]  Jasper A. Vrugt,et al.  Semi-distributed parameter optimization and uncertainty assessment for large-scale streamflow simulation using global optimization / Optimisation de paramètres semi-distribués et évaluation de l'incertitude pour la simulation de débits à grande échelle par l'utilisation d'une optimisation globale , 2008 .

[7]  J. Kirchner Getting the right answers for the right reasons: Linking measurements, analyses, and models to advance the science of hydrology , 2006 .

[8]  Xiaobao Li,et al.  Multiple Parameterization for Hydraulic Conductivity Identification , 2008, Ground water.

[9]  John Doherty,et al.  A short exploration of structural noise , 2010 .

[10]  M. Sivapalan,et al.  Process controls of water balance variability in a large semi-arid catchment: downward approach to hydrological model development , 2001 .

[11]  Yu-Shu Wu,et al.  Modeling Hydraulic Responses to Meteorological Forcing: From Canopy to Aquifer , 2008 .

[12]  George Kuczera,et al.  Calibration of conceptual hydrological models revisited: 2. Improving optimisation and analysis , 2006 .

[13]  T. E. Unny,et al.  Predicting groundwater flow in a phreatic aquifer , 1987 .

[14]  Hoshin Vijai Gupta,et al.  On the ability to infer spatial catchment variability using streamflow hydrographs , 2011 .

[15]  Jan Vanderborght,et al.  Proof of concept of regional scale hydrologic simulations at hydrologic resolution utilizing massively parallel computer resources , 2010 .

[16]  Velimir V. Vesselinov,et al.  Improved inverse modeling for flow and transport in subsurface media: Combined parameter and state estimation , 2005 .

[17]  D. Kavetski,et al.  Towards a Bayesian total error analysis of conceptual rainfall-runoff models: Characterising model error using storm-dependent parameters , 2006 .

[18]  Bruce A. Robinson,et al.  Treatment of uncertainty using ensemble methods: Comparison of sequential data assimilation and Bayesian model averaging , 2007 .

[19]  Gedeon Dagan,et al.  Solute Dispersion in Unsaturated Heterogeneous Soil at Field Scale: I. Theory , 1979 .

[20]  You‐Kuan Zhang Stochastic Methods for Flow in Porous Media: Coping with Uncertainties , 2001 .

[21]  Warren W. Wood It's the Heterogeneity! , 2000 .

[22]  Judea Pearl,et al.  Fusion, Propagation, and Structuring in Belief Networks , 1986, Artif. Intell..

[23]  John Bredehoeft Models and model analysis. , 2010, Ground water.

[24]  John D. Bredehoeft,et al.  Ground-water models cannot be validated , 1992 .

[25]  Ray Leuning,et al.  Diagnosing errors in a land surface model (CABLE) in the time and frequency domains , 2011 .

[26]  Hoshin Vijai Gupta,et al.  How Bayesian data assimilation can be used to estimate the mathematical structure of a model , 2010 .

[27]  Velimir V. Vesselinov,et al.  Three‐dimensional numerical inversion of pneumatic cross‐hole tests in unsaturated fractured tuff: 1. Methodology and borehole effects , 2001 .

[28]  C.A.J. Appelo,et al.  FLOW AND TRANSPORT , 2004 .

[29]  Z. Shipton,et al.  What do you think this is? "Conceptual uncertainty" in geoscience interpretation , 2007 .

[30]  P. Young,et al.  Simplicity out of complexity in environmental modelling: Occam's razor revisited. , 1996 .

[31]  Soroosh Sorooshian,et al.  Impact of field‐calibrated vegetation parameters on GCM climate simulations , 2001 .

[32]  Kevin W. Manning,et al.  The community Noah land surface model with multiparameterization options (Noah-MP): 1. Model description and evaluation with local-scale measurements , 2011 .

[33]  Keith Beven,et al.  Catchment geomorphology and the dynamics of runoff contributing areas , 1983 .

[34]  J. Randerson,et al.  Technical Description of version 4.0 of the Community Land Model (CLM) , 2010 .

[35]  Howard Wheater,et al.  Bayesian conditioning of a rainfall‐runoff model for predicting flows in ungauged catchments and under land use changes , 2011 .

[36]  Peter Molnar,et al.  John Perry's neglected critique of Kelvin's age for the Earth: A missed opportunity in geodynamics , 2007 .

[37]  Mary C Hill,et al.  The Practical Use of Simplicity in Developing Ground Water Models , 2006, Ground water.

[38]  Lev R Ginzburg,et al.  Rules of thumb for judging ecological theories. , 2004, Trends in ecology & evolution.

[39]  Kuolin Hsu,et al.  Uncertainty assessment of hydrologic model states and parameters: Sequential data assimilation using the particle filter , 2005 .

[40]  R. Stouffer,et al.  Stationarity Is Dead: Whither Water Management? , 2008, Science.

[41]  R. Hunt,et al.  Are Models Too Simple? Arguments for Increased Parameterization , 2007, Ground water.

[42]  Ming Ye,et al.  Maximum likelihood Bayesian averaging of spatial variability models in unsaturated fractured tuff , 2003 .

[43]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[44]  P. Young,et al.  Identification of nonlinearity in rainfall‐flow response using data‐based mechanistic modeling , 2011 .

[45]  G. Pinder,et al.  Computational Methods in Subsurface Flow , 1983 .

[46]  C. E. Desborough,et al.  Surface energy balance complexity in GCM land surface models , 1999 .

[47]  Luis A. Bastidas,et al.  Calibrating a land surface model of varying complexity using multicriteria methods and the Cabauw dataset , 2002 .

[48]  John S. Selker,et al.  On the use of the Boussinesq equation for interpreting recession hydrographs from sloping aquifers , 2006 .

[49]  K Beven,et al.  On the concept of model structural error. , 2005, Water science and technology : a journal of the International Association on Water Pollution Research.

[50]  T. Harter,et al.  Conditional stochastic analysis of solute transport in heterogeneous, variably saturated soils , 1996 .

[51]  Carol S. Woodward,et al.  Development of a Coupled Groundwater-Atmosphere Model , 2011 .

[52]  Vladimir Cvetkovic,et al.  The Effect of Heterogeneity on Large Scale Solute Transport in the Unsaturated Zone , 1989 .

[53]  Doug Nychka,et al.  Forecasting skill of model averages , 2010 .

[54]  Keith Beven,et al.  Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the GLUE methodology , 2001 .

[55]  L. Shawn Matott,et al.  Calibration of complex subsurface reaction models using a surrogate-model approach , 2008 .

[56]  Thomas Harter,et al.  Stochastic analysis of solute transport in heterogeneous, variably saturated soils , 1996 .

[57]  Murugesu Sivapalan,et al.  Pattern, Process and Function: Elements of a Unified Theory of Hydrology at the Catchment Scale , 2006 .

[58]  William A. Jury,et al.  Simulation of solute transport using a transfer function model , 1982 .

[59]  S. P. Neuman,et al.  Maximum likelihood Bayesian averaging of uncertain model predictions , 2003 .

[60]  P. Matgen,et al.  Understanding catchment behavior through stepwise model concept improvement , 2008 .

[61]  Eric F. Wood,et al.  Strategies for large-scale, distributed hydrologic simulation , 1988 .

[62]  George Kuczera,et al.  Model smoothing strategies to remove microscale discontinuities and spurious secondary optima in objective functions in hydrological calibration , 2007 .

[63]  Praveen Kumar,et al.  Typology of hydrologic predictability , 2011 .

[64]  Peter C. Young,et al.  Data-based mechanistic modelling of environmental, ecological, economic and engineering systems. , 1998 .

[65]  Yuqiong Liu,et al.  Reconciling theory with observations: elements of a diagnostic approach to model evaluation , 2008 .

[66]  S. Attinger,et al.  Multiscale parameter regionalization of a grid‐based hydrologic model at the mesoscale , 2010 .

[67]  Jasper A. Vrugt,et al.  One‐, two‐, and three‐dimensional root water uptake functions for transient modeling , 2001 .

[68]  Aldo Fiori,et al.  Stochastic analysis of transport in a combined heterogeneous vadose zone–groundwater flow system , 2009 .

[69]  J. Vrugt,et al.  Corruption of accuracy and efficiency of Markov chain Monte Carlo simulation by inaccurate numerical implementation of conceptual hydrologic models , 2010 .

[70]  Jirka Šimůnek,et al.  Modeling Nonequilibrium Flow and Transport Processes Using HYDRUS , 2008 .

[71]  Karsten Pruess,et al.  A Physically Based Approach for Modeling Multiphase Fracture-Matrix Interaction in Fractured Porous Media , 2004 .

[72]  Keith Beven,et al.  Towards an alternative blueprint for a physically based digitally simulated hydrologic response modelling system , 2002 .

[73]  Ming Ye,et al.  Numerical Evaluation of Uncertainty in Water Retention Parameters and Effect on Predictive Uncertainty , 2009 .

[74]  S. P. Neuman,et al.  Improved forward and inverse analyses of saturated‐unsaturated flow toward a well in a compressible unconfined aquifer , 2010 .

[75]  Mary C. Hill,et al.  MMA, A Computer Code for Multi-Model Analysis , 2014 .

[76]  Jasper A. Vrugt,et al.  Inverse modeling of cloud-aerosol interactions - Part 2: Sensitivity tests on liquid phase clouds using a Markov Chain Monte Carlo based simulation approach , 2011 .

[77]  Linda M. Abriola,et al.  Use of the Richards equation in land surface parameterizations , 1999 .

[78]  Zong-Liang Yang,et al.  Development of a simple groundwater model for use in climate models and evaluation with Gravity Recovery and Climate Experiment data , 2007 .

[79]  A. Saltelli,et al.  A quantitative model-independent method for global sensitivity analysis of model output , 1999 .

[80]  George Kuczera,et al.  Semidistributed hydrological modeling: A “saturation path” perspective on TOPMODEL and VIC , 2003 .

[81]  Murugesu Sivapalan,et al.  Climate, soil, and vegetation controls upon the variability of water balance in temperate and semiarid landscapes: Downward approach to water balance analysis , 2003 .

[82]  L. Gelhar Stochastic Subsurface Hydrology , 1992 .

[83]  David Anderson,et al.  Multimodel Ranking and Inference in Ground Water Modeling , 2004, Ground water.

[84]  M. Sivapalan,et al.  A unifying framework for watershed thermodynamics: balance equations for mass, momentum, energy and entropy, and the second law of thermodynamics , 1998 .

[85]  Gour-Tsyh Yeh,et al.  Computational Subsurface Hydrology: Fluid Flows , 1999 .

[86]  Peter A. Troch,et al.  Climate and vegetation water use efficiency at catchment scales , 2009 .

[87]  C. Zheng,et al.  Applied contaminant transport modeling , 2002 .

[88]  Frank T.-C. Tsai,et al.  Characterization and identification of aquifer heterogeneity with generalized parameterization and Bayesian estimation , 2004 .

[89]  주진철,et al.  Applied Contaminant Transport Modeling, Second Edition , 2009 .

[90]  K. Beven,et al.  Toward a generalization of the TOPMODEL concepts:Topographic indices of hydrological similarity , 1996 .

[91]  J. Kirchner Catchments as simple dynamical systems: Catchment characterization, rainfall‐runoff modeling, and doing hydrology backward , 2009 .

[92]  Dmitri Kavetski,et al.  Ancient numerical daemons of conceptual hydrological modeling: 1. Fidelity and efficiency of time stepping schemes , 2010 .

[93]  Mark D. White,et al.  Modeling fluid flow and transport in variably saturated porous media with the STOMP simulator. 2. Verification and validation exercises , 1995 .

[94]  Thomas A. McMahon,et al.  Physically based hydrologic modeling: 2. Is the concept realistic? , 1992 .

[95]  Jasper A. Vrugt,et al.  Multiresponse multilayer vadose zone model calibration using Markov chain Monte Carlo simulation and field water retention data , 2011 .

[96]  W. James Shuttleworth,et al.  A fully multiple-criteria implementation of the Sobol' method for parameter sensitivity analysis , 2012 .

[97]  Jasper A. Vrugt,et al.  Comparison of point forecast accuracy of model averaging methods in hydrologic applications , 2010 .

[98]  A. J. Desbarats,et al.  Subsurface Flow and Transport: A Stochastic Approach , 1998 .

[99]  T. E. Unny,et al.  Semigroup solutions to stochastic unsteady groundwater flow subject to random parameters , 1987 .

[100]  A. W. Harbaugh MODFLOW-2005 : the U.S. Geological Survey modular ground-water model--the ground-water flow process , 2005 .

[101]  Wesley W. Wallender,et al.  Inverse modeling of large-scale spatially distributed vadose zone properties using global optimization / W06503, doi:10.1029/2003WR002706 , 2004 .

[102]  Tao Yang,et al.  On the Granularity and Clustering of Directed Acyclic Task Graphs , 1993, IEEE Trans. Parallel Distributed Syst..

[103]  N Oreskes,et al.  Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences , 1994, Science.

[104]  Murugesu Sivapalan,et al.  Modelling water balances in an Alpine catchment through exploitation of emergent properties over changing time scales , 2003 .

[105]  Thomas Harter,et al.  Water flow and solute spreading in heterogeneous soils with spatially variable water content , 1999 .

[106]  Dong-Jun Seo,et al.  Towards the characterization of streamflow simulation uncertainty through multimodel ensembles , 2004 .

[107]  Victor Koren,et al.  Use of Soil Property Data in the Derivation of Conceptual Rainfall-Runoff Model Parameters , 2000 .

[108]  Hoshin V. Gupta,et al.  Hydrologic consistency as a basis for assessing complexity of monthly water balance models for the continental United States , 2011 .

[109]  Keith Beven,et al.  Searching for the Holy Grail of scientific hydrology: Q t =( S, R, Δt ) A as closure , 2006 .

[110]  C. Granger,et al.  Improved methods of combining forecasts , 1984 .

[111]  John L. Nieber,et al.  Regionalized drought flow hydrographs from a mature glaciated plateau , 1977 .

[112]  Karl-Josef Hollenbeck,et al.  Maximum-likelihood estimation of unsaturated hydraulic parameters , 1998 .

[113]  Peter C. Young,et al.  A unified approach to environmental systems modeling , 2009 .

[114]  Jean-Raynald de Dreuzy,et al.  A new particle‐tracking approach to simulating transport in heterogeneous fractured porous media , 2010 .

[115]  Keith Beven,et al.  On doing better hydrological science , 2008 .

[116]  Soroosh Sorooshian,et al.  Toward improved streamflow forecasts: value of semidistributed modeling , 2001 .

[117]  Peter C. Young,et al.  Top‐down and data‐based mechanistic modelling of rainfall–flow dynamics at the catchment scale , 2003 .

[118]  I. C. Prentice,et al.  A dynamic global vegetation model for studies of the coupled atmosphere‐biosphere system , 2005 .

[119]  Hoshin V. Gupta,et al.  On the use of spatial regularization strategies to improve calibration of distributed watershed models , 2010 .

[120]  Monica G. Turner,et al.  Cross–Scale Interactions and Changing Pattern–Process Relationships: Consequences for System Dynamics , 2007, Ecosystems.

[121]  J. Freer,et al.  Consistency between hydrological models and field observations: linking processes at the hillslope scale to hydrological responses at the watershed scale , 2009 .

[122]  M. Celia,et al.  A General Mass-Conservative Numerical Solution for the Unsaturated Flow Equation , 1990 .

[123]  L. Konikow The Secret to Successful Solute‐Transport Modeling , 2011, Ground water.

[124]  Ming Ye,et al.  A Model‐Averaging Method for Assessing Groundwater Conceptual Model Uncertainty , 2010, Ground water.

[125]  Karl-Josef Hollenbeck,et al.  Experimental evidence of randomness and nonuniqueness in unsaturated outflow experiments designed for hydraulic parameter estimation , 1998 .

[126]  A. J. Pitmana,et al.  The CHAmeleon Surface Model : description and use with the PILPS phase 2 ( e ) forcing data , 2003 .

[127]  Ming Ye,et al.  Expert elicitation of recharge model probabilities for the Death Valley regional flow system , 2008 .

[128]  Dennis L. Corwin,et al.  Applied Contaminant Transport Modeling, Second Edition , 2003 .

[129]  Alfred O. Hero,et al.  Geodesic entropic graphs for dimension and entropy estimation in manifold learning , 2004, IEEE Transactions on Signal Processing.

[130]  T. E. Unny,et al.  Random evolution equations in hydrology , 1990 .

[131]  J. Bredehoeft The conceptualization model problem—surprise , 2005 .

[132]  M. Clark,et al.  Ancient numerical daemons of conceptual hydrological modeling: 2. Impact of time stepping schemes on model analysis and prediction , 2010 .

[133]  H. Gupta,et al.  Correcting the mathematical structure of a hydrological model via Bayesian data assimilation , 2011 .

[134]  P. E. O'connell,et al.  IAHS Decade on Predictions in Ungauged Basins (PUB), 2003–2012: Shaping an exciting future for the hydrological sciences , 2003 .

[135]  M. Schaap,et al.  Simulation of field injection experiments in heterogeneous unsaturated media using cokriging and artificial neural network , 2007 .

[136]  W. Gray,et al.  A unifying framework for watershed thermodynamics: constitutive relationships , 1999 .

[137]  A. Gutjahr,et al.  Stochastic Analysis of Unsaturated Flow in Heterogeneous Soils: 2. Statistically Anisotropic Media With Variable α , 1985 .

[138]  David L. Alumbaugh,et al.  A geostatistically based inverse model for electrical resistivity surveys and its applications to vadose zone hydrology , 2002 .

[139]  Julian Klaus,et al.  Modelling rapid flow response of a tile‐drained field site using a 2D physically based model: assessment of ‘equifinal’ model setups , 2010 .

[140]  M. S. Hantush,et al.  Growth and Decay of Groundwater-Mounds in Response to Uniform Percolation , 1967 .

[141]  Allan L. Gutjahr,et al.  Stochastic Analysis of Unsaturated Flow in Heterogeneous Soils: 3. Observations and Applications , 1985 .

[142]  K. Beven Searching for the Holy Grail of Scientific Hydrology: Qt= H(S?R?)A as closure , 2006 .

[143]  Dmitri Kavetski,et al.  Pursuing the method of multiple working hypotheses for hydrological modeling , 2011 .

[144]  Murugesu Sivapalan,et al.  ON THE REPRESENTATIVE ELEMENTARY AREA (REA) CONCEPT AND ITS UTILITY FOR DISTRIBUTED RAINFALL-RUNOFF MODELLING , 1995 .

[145]  Jeroen P. van der Sluijs,et al.  A framework for dealing with uncertainty due to model structure error , 2004 .

[146]  M. Ye,et al.  A Markov chain model for characterizing medium heterogeneity and sediment layering structure , 2008 .

[147]  G. Dagan Flow and transport in porous formations , 1989 .

[148]  Mariano I. Cantero,et al.  Role of turbulence fluctuations on uncertainties of acoustic Doppler current profiler discharge measurements , 2012 .

[149]  Martyn P. Clark,et al.  Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models , 2008 .

[150]  George Kuczera,et al.  Calibration of conceptual hydrological models revisited: 1. Overcoming numerical artefacts , 2006 .

[151]  Michael Lehning,et al.  Meteorological Modeling of Very High-Resolution Wind Fields and Snow Deposition for Mountains , 2010 .

[152]  Martyn P. Clark,et al.  Rainfall‐runoff model calibration using informal likelihood measures within a Markov chain Monte Carlo sampling scheme , 2009 .

[153]  Alberto Guadagnini,et al.  Type‐curve estimation of statistical heterogeneity , 2004 .

[154]  Y. Rubin Applied Stochastic Hydrogeology , 2003 .

[155]  Hoshin Vijai Gupta,et al.  Multiple-criteria calibration of a distributed watershed model using spatial regularization and response signatures , 2012 .

[156]  K. Beven Rainfall-Runoff Modelling: The Primer , 2012 .

[157]  K. Ponnambalam,et al.  Stochastic partial differential equations in groundwater hydrology , 1991 .

[158]  George A Zyvoloski,et al.  An Investigation of Numerical Grid Effects in Parameter Estimation , 2006, Ground water.

[159]  Peter Reichert,et al.  Analyzing input and structural uncertainty of nonlinear dynamic models with stochastic, time‐dependent parameters , 2009 .

[160]  Chunmiao Zheng,et al.  MMA: A Computer Code for Multimodel Analysis , 2010 .

[161]  W. Kinzelbach Applied groundwater modeling — Simulation of flow and advective transport , 1992 .

[162]  Murugesu Sivapalan,et al.  Dominant physical controls on hourly flow predictions and the role of spatial variability: Mahurangi catchment, New Zealand , 2003 .

[163]  Dmitri Kavetski,et al.  Representing spatial variability of snow water equivalent in hydrologic and land‐surface models: A review , 2011 .

[164]  S. P. Neuman,et al.  On model selection criteria in multimodel analysis , 2007 .

[165]  Hoshin Vijai Gupta,et al.  Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling , 2009 .

[166]  Jeffrey J. McDonnell,et al.  On the dialog between experimentalist and modeler in catchment hydrology: Use of soft data for multicriteria model calibration , 2002 .

[167]  Soroosh Sorooshian,et al.  Dual state-parameter estimation of hydrological models using ensemble Kalman filter , 2005 .

[168]  Allan L. Gutjahr,et al.  Stochastic Analysis of Unsaturated Flow in Heterogeneous Soils: 1. Statistically Isotropic Media , 1985 .

[169]  S. P. Neuman,et al.  A comprehensive strategy of hydrogeologic modeling and uncertainty analysis for nuclear facilities a , 2003 .

[170]  Hoshin V. Gupta,et al.  Exploring the relationship between complexity and performance in a land surface model using the multicriteria method , 2002 .

[171]  T. Unny Stochastic partial differential equations in groundwater hydrology , 1989 .

[172]  V. Klemeš,et al.  Operational Testing of Hydrological Simulation Models , 2022 .

[173]  H. Gupta,et al.  Estimating the uncertain mathematical structure of a water balance model via Bayesian data assimilation , 2009 .

[174]  R. Ibbitt,et al.  Hydrological data assimilation with the ensemble Kalman filter: Use of streamflow observations to update states in a distributed hydrological model , 2007 .

[175]  Keith Beven,et al.  Changing ideas in hydrology — The case of physically-based models , 1989 .

[176]  S. Gorelick,et al.  Heterogeneity in Sedimentary Deposits: A Review of Structure‐Imitating, Process‐Imitating, and Descriptive Approaches , 1996 .

[177]  Randal D. Koster,et al.  Do Global Models Properly Represent the Feedback between Land and Atmosphere? , 2006, Journal of Hydrometeorology.

[178]  Patrick M. Reed,et al.  A top-down framework for watershed model evaluation and selection under uncertainty , 2009, Environ. Model. Softw..

[179]  M. Sivapalan,et al.  Improving model structure and reducing parameter uncertainty in conceptual water balance models through the use of auxiliary data , 2005 .

[180]  Hoshin Vijai Gupta,et al.  A spatial regularization approach to parameter estimation for a distributed watershed model , 2008 .

[181]  Vladimir Cvetkovic,et al.  Field scale mass arrival of sorptive solute into the groundwater , 1991 .

[182]  Robert Clement,et al.  On the validation of models of forest CO2 exchange using eddy covariance data: some perils and pitfalls. , 2005, Tree physiology.

[183]  Hoshin Vijai Gupta,et al.  On the development of regionalization relationships for lumped watershed models: The impact of ignoring sub-basin scale variability , 2009 .

[184]  Soroosh Sorooshian,et al.  Evaluating model performance and parameter behavior for varying levels of land surface model complexity , 2006 .

[185]  S. P. Neuman,et al.  Three‐dimensional numerical inversion of pneumatic cross‐hole tests in unsaturated fractured tuff: 2. Equivalent parameters, high‐resolution stochastic imaging and scale effects , 2001 .

[186]  C. Diks,et al.  Improved treatment of uncertainty in hydrologic modeling: Combining the strengths of global optimization and data assimilation , 2005 .

[187]  Stefano Tarantola,et al.  A Quantitative Model-Independent Method for Global Sensitivity Analysis of Model Output , 1999, Technometrics.

[188]  John D Bredehoeft From models to performance assessment: the conceptualization problem. , 2003, Ground water.

[189]  H. Kantz,et al.  Nonlinear time series analysis , 1997 .

[190]  Aaron Boone,et al.  Issues related to low resolution modeling of soil moisture: experience with the PLACE model , 1996 .

[191]  Paul Roman,et al.  The constructs of physics and the role of math--revisited , 1998 .

[192]  Mary P. Anderson,et al.  Introduction to Groundwater Modeling: Finite Difference and Finite Element Methods , 1982 .

[193]  Zhenghui Xie,et al.  A new parameterization for surface and groundwater interactions and its impact on water budgets with the variable infiltration capacity (VIC) land surface model , 2003 .