Calibration of Stochastic Computer Models Using Stochastic Approximation Methods

Computer models are widely used to simulate real processes. Within the computer model, there always exist some parameters which are unobservable in the real process but need to be specified in the model. The procedure to adjust these unknown parameters in order to fit the model to observed data and improve predictive capability is known as calibration. Practically, calibration is typically done manually. In this paper, we propose an effective and efficient algorithm based on the stochastic approximation (SA) approach that can be easily automated. We first demonstrate the feasibility of applying stochastic approximation to stochastic computer model calibration and apply it to three stochastic simulation models. We compare our proposed SA approach with another direct calibration search method, the genetic algorithm. The results indicate that our proposed SA approach performs equally as well in terms of accuracy and significantly better in terms of computational search time. We further consider the calibration parameter uncertainty in the subsequent application of the calibrated model and propose an approach to quantify it using asymptotic approximations.

[1]  Talal M. Alkhamis,et al.  Simulation-based optimization using simulated annealing with ranking and selection , 2002, Comput. Oper. Res..

[2]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[3]  Julie L. Swann,et al.  Simple Procedures for Selecting the Best Simulated System When the Number of Alternatives is Large , 2001, Oper. Res..

[4]  J. Spall Adaptive stochastic approximation by the simultaneous perturbation method , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[5]  Liudong Xing,et al.  Consequence Oriented Self-Healing and Autonomous Diagnosis for Highly Reliable Systems and Software , 2011, IEEE Transactions on Reliability.

[6]  Laura Painton Swiler,et al.  Calibration, validation, and sensitivity analysis: What's what , 2006, Reliab. Eng. Syst. Saf..

[7]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[8]  Soon-Thiam Khu,et al.  Using genetic algorithms to calibrate a water quality model. , 2007, The Science of the total environment.

[9]  Gang George Yin Rates of Convergence for a Class of Global Stochastic Optimization Algorithms , 1999, SIAM J. Optim..

[10]  H. Robbins A Stochastic Approximation Method , 1951 .

[11]  G Scott Gazelle,et al.  Calibration of disease simulation model using an engineering approach. , 2009, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[12]  Raghavan Srinivasan,et al.  Evaluation of global optimization algorithms for parameter calibration of a computationally intensive hydrologic model , 2009 .

[13]  Darren J. Wilkinson,et al.  Bayesian Emulation and Calibration of a Stochastic Computer Model of Mitochondrial DNA Deletions in Substantia Nigra Neurons , 2009 .

[14]  Xiaolong Wang,et al.  A Heuristic Algorithm for Reliability Modeling and Analysis of Grid Systems , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[15]  Rafael Martí Multi-Start Methods , 2003, Handbook of Metaheuristics.

[16]  Kaan Ozbay,et al.  New Calibration Methodology for Microscopic Traffic Simulation Using Enhanced Simultaneous Perturbation Stochastic Approximation Approach , 2009 .

[17]  Thomas J. Santner,et al.  Simultaneous Determination of Tuning and Calibration Parameters for Computer Experiments , 2009, Technometrics.

[18]  Yi Pan,et al.  A Hierarchical Modeling and Analysis for Grid Service Reliability , 2007, IEEE Transactions on Computers.

[19]  H. Michael Zhang,et al.  Calibration of Microsimulation with Heuristic Optimization Methods , 2007 .

[20]  R. Cheng,et al.  Sensitivity of computer simulation experiments to errors in input data , 1997 .

[21]  J. J. García-Escalante,et al.  Calibration of a greenhouse climate model using evolutionary algorithms , 2009 .

[22]  Eric Moulines,et al.  Stability of Stochastic Approximation under Verifiable Conditions , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[23]  J. Savarino,et al.  Bayesian Calibration of Microsimulation Models , 2009, Journal of the American Statistical Association.

[24]  A. OHagan,et al.  Bayesian analysis of computer code outputs: A tutorial , 2006, Reliab. Eng. Syst. Saf..

[25]  Haris N. Koutsopoulos,et al.  Calibration of Microscopic Traffic Simulation Models , 2007 .

[26]  L. Shawn Matott,et al.  Calibration of complex subsurface reaction models using a surrogate-model approach , 2008 .

[27]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[28]  Yong Chen,et al.  Robust Calibration for Localization in Clustered Wireless Sensor Networks , 2007, 2007 IEEE International Conference on Automation Science and Engineering.

[29]  J. Spall Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[30]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[31]  J. L. Maryak,et al.  Global random optimization by simultaneous perturbation stochastic approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[32]  Hatim O. Sharif,et al.  On the calibration and verification of two‐dimensional, distributed, Hortonian, continuous watershed models , 2000 .

[33]  Barry L. Nelson,et al.  Comparing Systems via Simulation , 2007 .

[34]  Moshe Leshno,et al.  The Prevalence Rate and Anatomic Location of Colorectal Adenoma and Cancer Detected by Colonoscopy in Average-Risk Individuals Aged 40–80 Years , 2006, The American Journal of Gastroenterology.

[35]  D. Bingham,et al.  Computer Model Calibration or Tuning in Practice , 2006 .