A performance comparison of sensitivity analysis methods for building energy models

The choice of sensitivity analysis methods for a model often relies on the behavior of model outputs. However, many building energy models are “black-box” functions whose behavior of simulated results is usually unknown or uncertain. This situation raises a question of how to correctly choose a sensitivity analysis method and its settings for building simulation. A performance comparison of nine sensitivity analysis methods has been carried out by means of computational experiments and building energy simulation. A comprehensive test procedure using three benchmark functions and two real-world building energy models was proposed. The degree of complexity was gradually increased by carefully-chosen test problems. Performance of these methods was compared through the ranking of variables’ importance, variables’ sensitivity indices, interaction among variables, and computational cost for each method. Test results show the consistency between the Fourier Amplitude Sensitivity Test (FAST) and the Sobol method. Some evidences found from the tests indicate that performance of other methods was unstable, especially with the non-monotonic test problems.

[1]  Kevin J. Lomas,et al.  Sensitivity analysis techniques for building thermal simulation programs , 1992 .

[2]  Wei Tian,et al.  A review of sensitivity analysis methods in building energy analysis , 2013 .

[3]  Kotek Petr,et al.  TECHNIQUE OF UNCERTAINTY AND SENSITIVITY ANALYSIS FOR SUSTAINABLE BUILDING ENERGY SYSTEMS PERFORMANCE CALCULATIONS , 2007 .

[4]  Anh Tuan Nguyen,et al.  Sustainable housing in Vietnam: Climate responsive design strategies to optimize thermal comfort , 2013 .

[5]  Sigrid Reiter,et al.  AN ADAPTIVE THERMAL COMFORT MODEL FOR HOT HUMID SOUTH- EAST ASIA , 2012 .

[6]  A. Saltelli,et al.  A quantitative model-independent method for global sensitivity analysis of model output , 1999 .

[7]  Michael S. Eldred,et al.  DAKOTA : a multilevel parallel object-oriented framework for design optimization, parameter estimation, uncertainty quantification, and sensitivity analysis. Version 5.0, user's reference manual. , 2010 .

[8]  Stefano Tarantola,et al.  Sensitivity Analysis in Practice , 2002 .

[9]  Stefano Tarantola,et al.  Application of global sensitivity analysis of model output to building thermal simulations , 2008 .

[10]  Jing Yang,et al.  Convergence and uncertainty analyses in Monte-Carlo based sensitivity analysis , 2011, Environ. Model. Softw..

[11]  Willy Bauwens,et al.  Sobol' sensitivity analysis of a complex environmental model , 2011, Environ. Model. Softw..

[12]  J. Kämpf,et al.  A comparison of global optimization algorithms with standard benchmark functions and real-world applications using EnergyPlus , 2009 .

[13]  D. Hamby.  A review of techniques for parameter sensitivity analysis of environmental models , 1994, Environmental monitoring and assessment.

[14]  Gianni Bellocchi,et al.  Comparison of sensitivity analysis techniques: A case study with the rice model WARM , 2010 .

[15]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[16]  I. Sobol,et al.  Sensitivity Measures, ANOVA-like Techniques and the Use of Bootstrap , 1997 .

[17]  Zheng O'Neill,et al.  Uncertainty and sensitivity decomposition of building energy models , 2012 .

[18]  Marc B. Neumann,et al.  Global sensitivity analysis in wastewater applications: A comprehensive comparison of different methods , 2013, Environ. Model. Softw..

[19]  I. Sobol,et al.  About the use of rank transformation in sensitivity analysis of model output , 1995 .

[20]  Richard Garber.  Optimisation Stories: The Impact of Building Information Modelling on Contemporary Design Practice , 2009 .

[21]  José Manuel Cejudo López,et al.  Uncertainties and sensitivity analysis in building energy simulation using macroparameters , 2013 .

[22]  M. Kendall,et al.  The Problem of $m$ Rankings , 1939 .

[23]  Sigrid Reiter,et al.  An investigation on thermal performance of a low cost apartment in hot humid climate of Danang , 2012 .

[24]  Jlm Jan Hensen,et al.  Uncertainty analysis in building performance simulation for design support , 2011 .

[25]  Stefan Finsterle,et al.  Modeling the performance of large-scale CO2 storage systems: A comparison of different sensitivity analysis methods , 2012 .

[26]  Guido Ridolfi.  Space systems conceptual design: Analysis methods for engineering-team support , 2013 .

[27]  A. Saltelli,et al.  An alternative way to compute Fourier amplitude sensitivity test (FAST) , 1998 .

[28]  Philippe Rigo,et al.  A review on simulation-based optimization methods applied to building performance analysis , 2014 .

[29]  I. Sobola,et al.  Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates , 2001 .

[30]  Joseph C. Lam,et al.  Sensitivity analysis of energy performance of office buildings , 1996 .

[31]  Ilya M. Sobol,et al.  Sensitivity Estimates for Nonlinear Mathematical Models , 1993 .

[32]  Henrik Brohus,et al.  Application of sensitivity analysis in design of sustainable buildings , 2009 .

[33]  Carolina Massmann,et al.  Analysis of the behavior of a rainfall-runoff model using three global sensitivity analysis methods evaluated at different temporal scales , 2012 .

[34]  F. J. Davis,et al.  Illustration of Sampling‐Based Methods for Uncertainty and Sensitivity Analysis , 2002, Risk analysis : an official publication of the Society for Risk Analysis.

[35]  Sophia Lefantzi,et al.  DAKOTA : a multilevel parallel object-oriented framework for design optimization, parameter estimation, uncertainty quantification, and sensitivity analysis. , 2011 .

[36]  K.,et al.  Nonlinear sensitivity analysis of multiparameter model systems , 1977 .

[37]  Christina J. Hopfe,et al.  Uncertainty and sensitivity analysis for detailed design support , 2007 .

[38]  Satish Narayanan,et al.  A COMPARATIVE STUDY ON UNCERTAINTY PROPAGATION IN HIGH PERFORMANCE BUILDING DESIGN , 2011 .

[39]  Michael S. Eldred,et al.  DAKOTA , A Multilevel Parallel Object-Oriented Framework for Design Optimization , Parameter Estimation , Uncertainty Quantification , and Sensitivity Analysis Version 4 . 0 User ’ s Manual , 2006 .

[40]  Jon C. Helton,et al.  A comparison of uncertainty and sensitivity analysis results obtained with random and Latin hypercube sampling , 2005, Reliab. Eng. Syst. Saf..

[41]  H. Christopher Frey,et al.  Evaluation of Selected Sensitivity Analysis Methods Based Upon Applications to Two Food Safety Process Risk Models , 2003 .

[42]  Andrea Saltelli,et al.  An effective screening design for sensitivity analysis of large models , 2007, Environ. Model. Softw..

[43]  Anh Tuan Nguyen,et al.  Passive designs and strategies for low-cost housing using simulation-based optimization and different thermal comfort criteria , 2014 .