What has Global Sensitivity Analysis ever done for us? A systematic review to support scientific advancement and to inform policy-making in earth system modelling

Abstract Computer models are essential tools in the earth system sciences. They underpin our search for understanding of earth system functioning and support decision- and policy-making across spatial and temporal scales. To understand the implications of uncertainty and environmental variability on the identification of such earth system models and their predictions, we can rely on increasingly powerful Global Sensitivity Analysis (GSA) methods. Previous reviews have characterised the variability of GSA methods available and their usability for different tasks. In our paper we rather focus on reviewing what has been learned so far by applying GSA to models across the earth system sciences, independently of the specific algorithm that was applied. We identify and discuss 10 key findings with general applicability and relevance for the earth sciences. We further provide an A-B-C-D of best practise in applying GSA methods, which we have derived from analysing why some GSA applications provided more insight than others.

[1]  Karline Soetaert,et al.  On the coupling of benthic and pelagic biogeochemical models , 2000 .

[2]  Jing Wang,et al.  Parameter sensitivity analysis of crop growth models based on the extended Fourier Amplitude Sensitivity Test method , 2013, Environ. Model. Softw..

[3]  Hilary McMillan,et al.  Uncertainty in hydrological signatures , 2015 .

[4]  Andrea Castelletti,et al.  Emulation techniques for the reduction and sensitivity analysis of complex environmental models , 2012, Environ. Model. Softw..

[5]  Mun-Ju Shin,et al.  Assessment of the suitability of rainfall-runoff models by coupling performance statistics and sensitivity analysis , 2017 .

[6]  S. Uhlenbrook,et al.  Sensitivity analyses of a distributed catchment model to verify the model structure , 2005 .

[7]  John Norton,et al.  An introduction to sensitivity assessment of simulation models , 2015, Environ. Model. Softw..

[8]  Stefano Tarantola,et al.  A General Probabilistic Framework for uncertainty and global sensitivity analysis of deterministic models: A hydrological case study , 2014, Environ. Model. Softw..

[9]  Matteo Giuliani,et al.  Policy tree optimization for threshold-based water resources management over multiple timescales , 2018, Environ. Model. Softw..

[10]  Ming Ye,et al.  Global sensitivity analysis in hydrological modeling: Review of concepts, methods, theoretical framework, and applications , 2015 .

[11]  Ming Ye,et al.  Towards a comprehensive assessment of model structural adequacy , 2012 .

[12]  M. K. Ramamurthy,et al.  A new generation of cyberinfrastructure and data services for earth system science education and research , 2006 .

[13]  Emanuele Borgonovo,et al.  Model emulation and moment-independent sensitivity analysis: An application to environmental modelling , 2012, Environ. Model. Softw..

[14]  Huan Wu,et al.  Stepwise sensitivity analysis from qualitative to quantitative: Application to the terrestrial hydrological modeling of a Conjunctive Surface‐Subsurface Process (CSSP) land surface model , 2015 .

[15]  Francesca Pianosi,et al.  Dealing with deep uncertainties in landslide modelling for disaster risk reduction under climate change , 2016 .

[16]  Wei Chu,et al.  A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrological model , 2014, Environ. Model. Softw..

[17]  Thorsten Wagener,et al.  Investigating controls on the thermal sensitivity of Pennsylvania streams , 2012 .

[18]  Bryan A. Tolson,et al.  Quantitative global sensitivity analysis of the RZWQM to warrant a robust and effective calibration , 2014 .

[19]  J. McDonnell,et al.  A decade of Predictions in Ungauged Basins (PUB)—a review , 2013 .

[20]  Benjamin P. Bryant,et al.  Thinking Inside the Box , 2010 .

[21]  Lawrence Buja,et al.  The computational future for climate and Earth system models: on the path to petaflop and beyond , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[22]  Declan Butler,et al.  Earth Monitoring: The planetary panopticon , 2007, Nature.

[23]  Francesca Pianosi,et al.  A Multimethod Global Sensitivity Analysis Approach to Support the Calibration and Evaluation of Land Surface Models , 2017 .

[24]  Jerome R. Ravetz,et al.  Uncertainty and Quality in Science for Policy , 1990 .

[25]  Thorsten Wagener,et al.  Testing the realism of model structures to identify karst system processes using water quality and quantity signatures , 2013 .

[26]  Neus Sabater,et al.  Emulation of Leaf, Canopy and Atmosphere Radiative Transfer Models for Fast Global Sensitivity Analysis , 2016, Remote. Sens..

[27]  Keith Beven,et al.  The future of distributed models: model calibration and uncertainty prediction. , 1992 .

[28]  Kirk Martinez,et al.  Environmental Sensor Networks: A revolution in the earth system science? , 2006 .

[29]  Marc B. Neumann,et al.  Global sensitivity analysis for urban water quality modelling: terminology, convergence and comparison of different methods. , 2015 .

[30]  Jim W. Hall,et al.  Sensitivity analysis of environmental models: A systematic review with practical workflow , 2014, Environ. Model. Softw..

[31]  P. Bates,et al.  Quantifying the importance of spatial resolution and other factors through global sensitivity analysis of a flood inundation model , 2016 .

[32]  Patrick M. Reed,et al.  Advancing the identification and evaluation of distributed rainfall‐runoff models using global sensitivity analysis , 2007 .

[33]  R. Peng Reproducible Research in Computational Science , 2011, Science.

[34]  Stefano Tarantola,et al.  Trends in sensitivity analysis practice in the last decade. , 2016, The Science of the total environment.

[35]  M. Moss Some basic considerations in the design of hydrologic data networks , 1979 .

[36]  N. Fohrer,et al.  How to improve the representation of hydrological processes in SWAT for a lowland catchment – temporal analysis of parameter sensitivity and model performance , 2014 .

[37]  R. Stouffer,et al.  Stationarity Is Dead: Whither Water Management? , 2008, Science.

[38]  Steven C Bankes,et al.  Tools and techniques for developing policies for complex and uncertain systems , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[39]  M. Ek,et al.  Hyperresolution global land surface modeling: Meeting a grand challenge for monitoring Earth's terrestrial water , 2011 .

[40]  J. Kirchner Getting the right answers for the right reasons: Linking measurements, analyses, and models to advance the science of hydrology , 2006 .

[41]  Bruce E. Jaffe,et al.  Effects of fringing reefs on tsunami inundation: American Samoa , 2009 .

[42]  Murray C. Peel,et al.  Simulating runoff under changing climatic conditions: Revisiting an apparent deficiency of conceptual rainfall‐runoff models , 2016 .

[43]  George Kuczera,et al.  Toward a reliable decomposition of predictive uncertainty in hydrological modeling: Characterizing rainfall errors using conditional simulation , 2011 .

[44]  J. Lundquist,et al.  Exploring the impact of forcing error characteristics on physically based snow simulations within a global sensitivity analysis framework , 2014 .

[45]  P. J. Smith,et al.  A novel framework for discharge uncertainty quantification applied to 500 UK gauging stations , 2015, Water resources research.

[46]  Soroosh Sorooshian,et al.  A framework for development and application of hydrological models , 2001, Hydrology and Earth System Sciences.

[47]  George Kuczera,et al.  Understanding predictive uncertainty in hydrologic modeling: The challenge of identifying input and structural errors , 2010 .

[48]  Patrick M. Reed,et al.  Time‐varying sensitivity analysis clarifies the effects of watershed model formulation on model behavior , 2013 .

[49]  Jeremy Rohmer,et al.  Evaluating uncertainties of future marine flooding occurrence as sea-level rises , 2015, Environ. Model. Softw..

[50]  Thorsten Wagener,et al.  Model‐based analysis of the influence of catchment properties on hydrologic partitioning across five mountain headwater subcatchments , 2015, Water resources research.

[51]  F. Ackerman,et al.  Inside the integrated assessment models: Four issues in climate economics , 2009 .

[52]  Emanuele Borgonovo,et al.  Making the most out of a hydrological model data set: Sensitivity analyses to open the model black‐box , 2017 .

[53]  P. Reed,et al.  Hydrology and Earth System Sciences Discussions Comparing Sensitivity Analysis Methods to Advance Lumped Watershed Model Identification and Evaluation , 2022 .

[54]  H J van Meerveld,et al.  When should stream water be sampled to be most informative for event-based, multi-criteria model calibration? , 2017 .

[55]  M. Santosh,et al.  Supercontinents, mantle dynamics and plate tectonics: A perspective based on conceptual vs. numerical models , 2011 .

[56]  Suraje Dessai,et al.  Robust adaptation to climate change , 2010 .

[57]  K. Beven,et al.  Comment on “Hyperresolution global land surface modeling: Meeting a grand challenge for monitoring Earth's terrestrial water” by Eric F. Wood et al. , 2012 .

[58]  Vivien Mallet,et al.  Emulation and Sobol' sensitivity analysis of an atmospheric dispersion model applied to the Fukushima nuclear accident , 2016 .

[59]  Francesca Pianosi,et al.  Global Sensitivity Analysis of environmental models: Convergence and validation , 2016, Environ. Model. Softw..

[60]  Augusto Neri,et al.  Uncertainty quantification and sensitivity analysis of volcanic columns models: results from the integral model PLUME-MoM , 2016 .

[61]  Andreas Hartmann,et al.  On the value of water quality data and informative flow states in karst modelling , 2017 .

[62]  Roberto Confalonieri,et al.  Sensitivity analysis of a sensitivity analysis: We are likely overlooking the impact of distributional assumptions , 2016 .

[63]  Peter A. Troch,et al.  The future of hydrology: An evolving science for a changing world , 2010 .

[64]  Peter Steen Mikkelsen,et al.  Application of global sensitivity analysis and uncertainty quantification in dynamic modelling of micropollutants in stormwater runoff , 2012, Environ. Model. Softw..

[65]  Bryan A. Tolson,et al.  Optimizing hydrological consistency by incorporating hydrological signatures into model calibration objectives , 2015 .

[66]  K. Beven,et al.  Bayesian Estimation of Uncertainty in Runoff Prediction and the Value of Data: An Application of the GLUE Approach , 1996 .

[67]  P. Reed,et al.  Rainfall characteristics define the value of streamflow observations for distributed watershed model identification , 2008 .

[68]  Hubert H. G. Savenije,et al.  Process consistency in models: The importance of system signatures, expert knowledge, and process complexity , 2014 .

[69]  Dominik E. Reusser,et al.  Process verification of a hydrological model using a temporal parameter sensitivity analysis , 2015 .

[70]  A. R. Ravishankara,et al.  A sensitivity analysis of key natural factors in the modeled global acetone budget , 2017 .

[71]  Eric A. Treml,et al.  Identifying the key biophysical drivers, connectivity outcomes, and metapopulation consequences of larval dispersal in the sea , 2015, Movement ecology.

[72]  E. Sudicky,et al.  Hyper‐resolution global hydrological modelling: what is next? , 2015 .

[73]  Stefano Nativi,et al.  Big Data challenges in building the Global Earth Observation System of Systems , 2015, Environ. Model. Softw..

[74]  V. Klemeš,et al.  Dilettantism in hydrology: Transition or destiny? , 1986 .

[75]  Francesca Pianosi,et al.  Comparison of variance-based and moment-independent global sensitivity analysis approaches by application to the SWAT model , 2017, Environ. Model. Softw..

[76]  H. Gupta,et al.  A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory , 2016 .

[77]  Francesca Pianosi,et al.  Understanding the time‐varying importance of different uncertainty sources in hydrological modelling using global sensitivity analysis , 2016 .

[78]  Yuqiong Liu,et al.  Reconciling theory with observations: elements of a diagnostic approach to model evaluation , 2008 .

[79]  William R. Cotton,et al.  Cloud venting — A review and some new global annual estimates , 1995 .

[80]  Christopher Hutton,et al.  Most computational hydrology is not reproducible, so is it really science? , 2016, Water Resources Research.

[81]  P. Reed,et al.  Sensitivity-guided reduction of parametric dimensionality for multi-objective calibration of watershed models , 2009 .

[82]  Thorsten Wagener,et al.  A vulnerability driven approach to identify adverse climate and land use change combinations for critical hydrologic indicator thresholds: Application to a watershed in Pennsylvania, USA , 2014 .

[83]  John P. Norton Algebraic sensitivity analysis of environmental models , 2008, Environ. Model. Softw..

[84]  Upmanu Lall,et al.  Floods in a changing climate: Does the past represent the future? , 2001, Water Resources Research.

[85]  Joseph R. Kasprzyk,et al.  Many objective robust decision making for complex environmental systems undergoing change , 2012, Environ. Model. Softw..

[86]  Hoshin Vijai Gupta,et al.  Model identification for hydrological forecasting under uncertainty , 2005 .

[87]  P. Reed,et al.  Characterization of watershed model behavior across a hydroclimatic gradient , 2008 .

[88]  Lei Chen,et al.  Time-varying sensitivity analysis of hydrologic and sediment parameters at multiple timescales: Implications for conservation practices. , 2017, The Science of the total environment.

[89]  Felipe J. Colón-González,et al.  Multimodel assessment of water scarcity under climate change , 2013, Proceedings of the National Academy of Sciences.

[90]  C. Taylor,et al.  Stern Review: The Economics of Climate Change , 2006 .

[91]  Arnaud J. A. M. Temme,et al.  LORICA - A new model for linking landscape and soil profile evolution: Development and sensitivity analysis , 2016, Comput. Geosci..

[92]  S. Sorooshian,et al.  Automatic calibration of conceptual rainfall-runoff models: sensitivity to calibration data , 1996 .

[93]  Paul D. Bates,et al.  Distributed Sensitivity Analysis of Flood Inundation Model Calibration , 2005 .

[94]  Brett A. Bryan,et al.  Variance-based sensitivity analysis of a forest growth model , 2012 .

[95]  Andrea Saltelli,et al.  Sensitivity analysis didn't help. A practitioner's critique of the Stern review , 2010 .

[96]  Augusto Neri,et al.  PLUME-MoM 1.0: A new integral model of volcanic plumes based on the method of moments , 2015 .

[97]  Albert J. Kettner,et al.  Uncertainty and Sensitivity in Surface Dynamics Modeling , 2016, Comput. Geosci..

[98]  Philippe Gourbesville,et al.  Spatial Global Sensitivity Analysis of High Resolution classified topographic data use in 2D urban flood modelling , 2016, Environ. Model. Softw..

[99]  Alan O'Neill,et al.  The computational challenges of Earth-system science , 2002, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[100]  Nebojsa Nakicenovic,et al.  Characterizing Climate-Change Uncertainties for Decision-Makers , 2004 .

[101]  Thomas A. McMahon,et al.  Physically based hydrologic modeling: 2. Is the concept realistic? , 1992 .

[102]  Neil McIntyre,et al.  Towards reduced uncertainty in conceptual rainfall‐runoff modelling: dynamic identifiability analysis , 2003 .

[103]  W. James Shuttleworth,et al.  A fully multiple-criteria implementation of the Sobol' method for parameter sensitivity analysis , 2012 .

[104]  Vincent Barra,et al.  Screening parameters in the Pasture Simulation model using the Morris method , 2013 .

[105]  Fiona Tummon,et al.  Earth System Science Frontiers: An Early Career Perspective , 2017 .

[106]  Dmitri Kavetski,et al.  Practical Use of Computationally Frugal Model Analysis Methods , 2016, Ground water.

[107]  Hubert H. G. Savenije,et al.  A framework to assess the realism of model structures using hydrological signatures , 2012 .

[108]  N Oreskes,et al.  Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences , 1994, Science.

[109]  Stefano Tarantola,et al.  Sensitivity analysis of the rice model WARM in Europe: Exploring the effects of different locations, climates and methods of analysis on model sensitivity to crop parameters , 2010, Environ. Model. Softw..

[110]  Zong-Liang Yang,et al.  Quantifying parameter sensitivity, interaction, and transferability in hydrologically enhanced versions of the Noah land surface model over transition zones during the warm season , 2010 .

[111]  Patrick M. Reed,et al.  Identifying parametric controls and dependencies in integrated assessment models using global sensitivity analysis , 2014, Environ. Model. Softw..

[112]  Soroosh Sorooshian,et al.  Sensitivity analysis of a land surface scheme using multicriteria methods , 1999 .

[113]  Erwin Zehe,et al.  Inferring model structural deficits by analyzing temporal dynamics of model performance and parameter sensitivity , 2011 .

[114]  B. Croke,et al.  Addressing ten questions about conceptual rainfall–runoff models with global sensitivity analyses in R , 2013 .

[115]  Erwin Zehe,et al.  Temporal dynamics of model parameter sensitivity for computationally expensive models with the Fourier amplitude sensitivity test , 2011 .

[116]  C. Massmann,et al.  Analysing the Sub-processes of a Conceptual Rainfall-Runoff Model Using Information About the Parameter Sensitivity and Variance , 2015, Environmental Modeling & Assessment.

[117]  J. Freer,et al.  Benchmarking observational uncertainties for hydrology: rainfall, river discharge and water quality , 2012 .

[118]  Keith Beven,et al.  Towards integrated environmental models of everywhere: uncertainty, data and modelling as a learning process , 2007 .

[119]  K. Beven Towards a coherent philosophy for modelling the environment , 2002, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[120]  H. Gupta,et al.  Understanding uncertainty in distributed flash flood forecasting for semiarid regions , 2008 .

[121]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .

[122]  Silvio Funtowicz,et al.  Are all models wrong? , 2020, Computational and systems oncology.

[123]  Cosimo Solidoro,et al.  Global sensitivity analysis of a shallow-water 3D eutrophication model , 1999 .

[124]  Casey Brown,et al.  Decision scaling: Linking bottom‐up vulnerability analysis with climate projections in the water sector , 2012 .

[125]  Hoshin V. Gupta,et al.  Hydrologic consistency as a basis for assessing complexity of monthly water balance models for the continental United States , 2011 .