Application of a variance‐based sensitivity analysis method to the Biomass Scenario Learning Model

Variance‐based sensitivity analysis can provide a comprehensive understanding of the input factors that drive model behavior, complementing more traditional system dynamics methods with quantitative metrics. This paper presents the methodology of a variance‐based sensitivity analysis of the Biomass Scenario Learning Model, a published STELLA model of interactions among investment, production, and learning in an emerging competitive industry. We document the methodology requirements, interpretations, and constraints, and compute estimated sensitivity indices and their uncertainties. We show that application of variance‐based sensitivity analysis to the model allows us to test for non‐additivity, identify influential and interactive variables, and confirm model formulation. To enable use of this type of sensitivity analysis in other system dynamics models, we provide this study's R code, annotated to facilitate adaptation to other studies. A related paper describes application of these techniques to the much larger Biomass Scenario Model.

[1]  Stefano Tarantola,et al.  Estimating the approximation error when fixing unessential factors in global sensitivity analysis , 2007, Reliab. Eng. Syst. Saf..

[2]  Andrew Ford,et al.  Statistical screening of system dynamics models , 2005 .

[3]  Paola Annoni,et al.  Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index , 2010, Comput. Phys. Commun..

[4]  I. Sobol,et al.  Sensitivity Measures, ANOVA-like Techniques and the Use of Bootstrap , 1997 .

[5]  A. Saltelli,et al.  Making best use of model evaluations to compute sensitivity indices , 2002 .

[6]  Ken R. McNaught,et al.  Using Morris' randomized OAT design as a factor screening method for developing simulation metamodels , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[7]  M. Jansen.  Analysis of variance designs for model output , 1999 .

[8]  Timothy R. B. Taylor,et al.  Improving model understanding using statistical screening , 2010 .

[9]  Brian Bush,et al.  Dynamic Modeling of Learning in Emerging Energy Industries: The Example of Advanced Biofuels in the United States: Preprint , 2015 .

[10]  Bob Walrave.  Determining intervention thresholds that change output behavior patterns , 2016 .

[11]  Asmeret Brooke Bier.  Copy of Sensitivity Analysis Techniques for System Dynamics Models of Human Behavior. , 2011 .

[12]  Y. Lin,et al.  Biomass Scenario Model Documentation: Data and References , 2013 .

[13]  Andrea Saltelli,et al.  Settings and methods for global sensitivity analysis - a short guide , 2007 .

[14]  Yaman Barlas,et al.  Sensitivity analysis for models with multiple behavior modes: a method based on behavior pattern measures , 2016 .

[15]  Jack P. C. Kleijnen,et al.  Sensitivity analysis and optimization of system dynamics models: Regression analysis and statistical design of experiments , 1995 .

[16]  Gönenç Yücel,et al.  Sensitivity analysis of graphical functions , 2014 .

[17]  David N. Ford,et al.  Quantifying the impacts of rework, schedule pressure, and ripple effect loops on project schedule performance , 2016 .

[18]  Jim W. Hall,et al.  Sensitivity analysis of environmental models: A systematic review with practical workflow , 2014, Environ. Model. Softw..

[19]  A. Saltelli,et al.  Importance measures in global sensitivity analysis of nonlinear models , 1996 .