Multicriteria sensitivity analysis as a diagnostic tool for understanding model behaviour and characterizing model uncertainty

Complex hydrological models are being increasingly used nowadays for many purposes such as studying the impact of climate and land-use change on water resources. However, building a high-fidelity model, particularly at large scales, remains a challenging task, due to complexities in model functioning and behavior and uncertainties in model structure, parameterization, and data. Global Sensitivity Analysis (GSA), which characterizes how the variation in the model response is attributed to variations in its input factors (e.g., parameters, forcing data), provides an opportunity to enhance the development and application of these complex models. In this paper, we advocate using GSA as an integral part of the modelling process by discussing its capabilities as a tool for diagnosing model structure and detecting potential defects, identifying influential factors, characterizing uncertainty, and selecting calibration parameters. Accordingly, we conduct a comprehensive GSA of a complex land surface-hydrology model, Modelisation Environmentale–Surface et Hydrologie (MESH), which combines the Canadian Land Surface Scheme (CLASS) with a hydrological routing component, WATROUTE. Various GSA experiments are carried out using a new technique, called Variogram Analysis of Response Surfaces (VARS), for alternative hydroclimatic conditions in Canada using multiple criteria, various model configurations, and a full set of model parameters. Results from this study reveal that, in addition to different hydroclimatic conditions and SA criteria, model configurations can also have a major impact on the assessment of sensitivity. GSA can identify aspects of the model internal functioning that are counter-intuitive, and thus, help the modeler to diagnose possible model deficiencies and make recommendations for improving development and application of the model. As a specific outcome of this work, a list of the most influential parameters for the MESH model is developed. This list, along with some specific recommendations, is expected to assist the wide community of MESH and CLASS users, to enhance their modelling applications.

[1]  D. Lawrence,et al.  Improving the representation of hydrologic processes in Earth System Models , 2015 .

[2]  Amin Haghnegahdar,et al.  An Improved Framework for Watershed Discretization and Model Calibration: Application to the Great Lakes Basin , 2015 .

[3]  R. Moore.  The PDM rainfall-runoff model , 2007 .

[4]  T. Hengl,et al.  Mapping the global depth to bedrock for land surface modeling , 2017 .

[5]  P. Reed,et al.  Characterization of watershed model behavior across a hydroclimatic gradient , 2008 .

[6]  M. Webb,et al.  Quantification of modelling uncertainties in a large ensemble of climate change simulations , 2004, Nature.

[7]  Joe R. Melton,et al.  Competition between plant functional types in the Canadian Terrestrial Ecosystem Model (CTEM) v. 2.0 , 2015 .

[8]  H. Gupta,et al.  A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory , 2016 .

[9]  Yuqiong Liu,et al.  Reconciling theory with observations: elements of a diagnostic approach to model evaluation , 2008 .

[10]  Paola Annoni,et al.  Sixth International Conference on Sensitivity Analysis of Model Output How to avoid a perfunctory sensitivity analysis , 2010 .

[11]  J. Mahfouf,et al.  A Canadian precipitation analysis (CaPA) project: Description and preliminary results , 2007 .

[12]  Alain Pietroniro,et al.  Development of the MESH modelling system for hydrological ensemble forecasting of the Laurentian Great Lakes at the regional scale , 2006 .

[13]  D. Lawrence,et al.  Parameterization improvements and functional and structural advances in Version 4 of the Community Land Model , 2011 .

[14]  Saman Razavi,et al.  Progressive Latin Hypercube Sampling: An efficient approach for robust sampling-based analysis of environmental models , 2017, Environ. Model. Softw..

[15]  A. Staniforth,et al.  The Operational CMC–MRB Global Environmental Multiscale (GEM) Model. Part II: Results , 1998 .

[16]  Laxmi Sushama,et al.  On the Arctic near-surface permafrost and climate sensitivities to soil and snow model formulations in climate models , 2014, Climate Dynamics.

[17]  Vivek K. Arora,et al.  Simulating energy and carbon fluxes over winter wheat using coupled land surface and terrestrial ecosystem models , 2003 .

[18]  Ming Ye,et al.  Global sensitivity analysis in hydrological modeling: Review of concepts, methods, theoretical framework, and applications , 2015 .

[19]  Richard W. Healy,et al.  Factors influencing ground-water recharge in the eastern United States , 2007 .

[20]  N. Kouwen,et al.  WATFLOOD: a Micro-Computer Based Flood Forecasting System Based on Real-Time Weather Radar , 1988 .

[21]  Joe R. Melton,et al.  The sensitivity of simulated competition between different plant functional types to subgrid‐scale representation of vegetation in a land surface model , 2014 .

[22]  Peter S. Eagleson.  The Emergence of Global‐Scale Hydrology , 1986 .

[23]  Erwin Zehe,et al.  Inferring model structural deficits by analyzing temporal dynamics of model performance and parameter sensitivity , 2011 .

[24]  N. Kouwen,et al.  Towards closing the vertical water balance in Canadian atmospheric models: Coupling of the land surface scheme class with the distributed hydrological model watflood , 2000 .

[25]  David M. Lawrence,et al.  Sensitivity of a model projection of near‐surface permafrost degradation to soil column depth and representation of soil organic matter , 2008 .

[26]  Saman Razavi,et al.  Insights into sensitivity analysis of Earth and environmental systems models: On the impact of parameter perturbation scale , 2017, Environ. Model. Softw..

[27]  D. Verseghy,et al.  CLASS-A Canadian Land Surface Scheme for GCMs , 1993 .

[28]  Robert S. Webb,et al.  Global Soil Texture and Derived Water-Holding Capacities (Webb et al.) , 2000 .

[29]  Saman Razavi,et al.  What do we mean by sensitivity analysis? The need for comprehensive characterization of “global” sensitivity in Earth and Environmental systems models , 2015 .

[30]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[31]  Philip Marsh,et al.  Characterizing snowmelt variability in a land‐surface‐hydrologic model , 2006 .

[32]  Dmitri Kavetski,et al.  Pursuing the method of multiple working hypotheses for hydrological modeling , 2011 .

[33]  Eric D. Soulis,et al.  A hydrology modelling framework for the Mackenzie GEWEX programme , 2003 .

[34]  V. Arora,et al.  Sub-grid scale representation of vegetation in global land surface schemes: implications for estimation of the terrestrial carbon sink , 2013 .

[35]  Alain Pietroniro,et al.  Grouped Response Units for Distributed Hydrologic Modeling , 1993 .

[36]  Bryan A. Tolson,et al.  Assimilation of SMOS soil moisture over the Great Lakes basin , 2015 .

[37]  Willy Bauwens,et al.  Multi-variable sensitivity and identifiability analysis for a complex environmental model in view of integrated water quantity and water quality modeling. , 2012, Water science and technology : a journal of the International Association on Water Pollution Research.

[38]  J. Kimball,et al.  A new global river network database for macroscale hydrologic modeling , 2012 .

[39]  Alain Pietroniro,et al.  Effects of Spatial Aggregation of Initial Conditions and Forcing Data on Modeling Snowmelt Using a Land Surface Scheme , 2008 .

[40]  Alain Pietroniro,et al.  Towards an improved land surface scheme for prairie landscapes , 2014 .

[41]  S. Uhlenbrook,et al.  Sensitivity analyses of a distributed catchment model to verify the model structure , 2005 .

[42]  Stefano Tarantola,et al.  A General Probabilistic Framework for uncertainty and global sensitivity analysis of deterministic models: A hydrological case study , 2014, Environ. Model. Softw..

[43]  Robert Leconte,et al.  What is Missing from the Prescription of Hydrology for Land Surface Schemes , 2016 .

[44]  Peter D. Blanken,et al.  Predicting the Net Basin Supply to the Great Lakes with a Hydrometeorological Model , 2012 .

[45]  Noel A Cressie,et al.  Statistics for Spatial Data. , 1992 .

[46]  Sabine Attinger,et al.  Computationally inexpensive identification of noninformative model parameters by sequential screening , 2015 .

[47]  Vivek K. Arora,et al.  A Representation of Variable Root Distribution in Dynamic Vegetation Models , 2003 .

[48]  Paulin Coulibaly,et al.  Application of SMOS Soil Moisture and Brightness Temperature at High Resolution With a Bias Correction Operator , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[49]  James R. Craig,et al.  Assessing the performance of a semi-distributed hydrological model under various watershed discretization schemes , 2015 .

[50]  Jim W. Hall,et al.  Sensitivity analysis of environmental models: A systematic review with practical workflow , 2014, Environ. Model. Softw..

[51]  C. Spence,et al.  Testing the ability of a semidistributed hydrological model to simulate contributing area , 2016 .

[52]  Michael Bruen,et al.  Impact of a physically based soil water flow and soil-plant interaction representation for modeling large-scale land surface processes , 2002 .

[53]  D. Verseghy,et al.  Class—A Canadian land surface scheme for GCMS. I. Soil model , 2007 .

[54]  Philip Marsh,et al.  Modelling the spatial–temporal variability of spring snowmelt in an arctic catchment , 2006 .

[55]  W. James Shuttleworth,et al.  A fully multiple-criteria implementation of the Sobol' method for parameter sensitivity analysis , 2012 .

[56]  J. Pelletier,et al.  A gridded global data set of soil, intact regolith, and sedimentary deposit thicknesses for regional and global land surface modeling , 2016 .

[57]  A. Ganji,et al.  Investigation of the 2013 Alberta flood from weather and climate perspectives , 2017, Climate Dynamics.

[58]  Harold Ritchie,et al.  ON THE USE OF COUPLED ATMOSPHERIC AND HYDROLOGIC MODELS AT REGIONAL SCALE , 2022 .

[59]  Shusen Wang,et al.  Modelling plant carbon and nitrogen dynamics of a boreal aspen forest in CLASS — the Canadian Land Surface Scheme , 2001 .

[60]  Balaji Rajagopalan,et al.  Are we unnecessarily constraining the agility of complex process‐based models? , 2015 .

[61]  Soroosh Sorooshian,et al.  Sensitivity analysis of a land surface scheme using multicriteria methods , 1999 .

[62]  Patrick M. Reed,et al.  From maps to movies: High-resolution time-varying sensitivity analysis for spatially distributed watershed models , 2013 .

[63]  A. Ganji,et al.  On improving cold region hydrological processes in the Canadian Land Surface Scheme , 2015, Theoretical and Applied Climatology.

[64]  Howard S. Wheater,et al.  Seasonal and extreme precipitation characteristics for the watersheds of the Canadian Prairie Provinces as simulated by the NARCCAP multi-RCM ensemble , 2014, Climate Dynamics.

[65]  I. Sobol.  Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates , 2001 .

[66]  Bart Nijssen,et al.  Monte Carlo sensitivity analysis of land surface parameters using the Variable Infiltration Capacity model , 2007 .

[67]  Fuxing Wang,et al.  The improvement of soil thermodynamics and its effects on land surface meteorology in the IPSL climate model , 2015 .

[68]  L. Shawn Matott,et al.  Calibrating Environment Canada's MESH Modelling System over the Great Lakes Basin , 2014 .

[69]  H. Gupta,et al.  A new framework for comprehensive, robust, and efficient global sensitivity analysis: 2. Application , 2016 .

[70]  K. Verdin,et al.  New Global Hydrography Derived From Spaceborne Elevation Data , 2008 .