Simulation model calibration with correlated knowledge-gradients

We address the problem of calibrating an approximate dynamic programming model, where we need to find a vector of parameters to produce the best fit of the model against historical data. The problem requires adaptively choosing the sequence of parameter settings on which to run the model, where each run of the model requires approximately twelve hours of CPU time and produces noisy non-stationary output. We describe an application of the knowledge-gradient algorithm with correlated beliefs to this problem and show that this algorithm finds a good parameter vector out of a population of one thousand with only three runs of the model.

[1]  Warren B. Powell,et al.  An Approximate Dynamic Programming Algorithm for Large-Scale Fleet Management: A Case Application , 2009, Transp. Sci..

[2]  Xi-Ren Cao.  Stochastic Learning and Optimization , 2007 .

[3]  Chun-Hung Chen,et al.  A gradient approach for smartly allocating computing budget for discrete event simulation , 1996, Winter Simulation Conference.

[4]  Warren B. Powell,et al.  The Knowledge-Gradient Policy for Correlated Normal Beliefs , 2009, INFORMS J. Comput..

[5]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[6]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[7]  Warren B. Powell,et al.  Asymptotic Optimality of Sequential Sampling Policies for Bayesian Information Collection , 2008 .

[8]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[9]  Chun-Hung Chen,et al.  Computing efforts allocation for ordinal optimization and discrete event simulation , 2000, IEEE Trans. Autom. Control..

[10]  Noel A Cressie,et al.  Statistics for Spatial Data, Revised Edition. , 1994 .

[11]  Ronald A. Howard,et al.  Information Value Theory , 1966, IEEE Trans. Syst. Sci. Cybern..

[12]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[13]  S. Gupta,et al.  Bayesian look ahead one-stage sampling allocations for selection of the best population , 1996 .

[14]  Chun-Hung Chen,et al.  Optimal computing budget allocation under correlated sampling , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[15]  Warren B. Powell,et al.  A Knowledge-Gradient Policy for Sequential Information Collection , 2008, SIAM J. Control. Optim..

[16]  Tim Hesterberg,et al.  Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control , 2004, Technometrics.

[17]  Chun-Hung Chen,et al.  An effective approach to smartly allocate computing budget for discrete event simulation , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[18]  Chun-Hung Chen,et al.  Computing budget allocation for simulation experiments with different system structures , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[19]  J. Mockus.  Bayesian Approach to Global Optimization: Theory and Applications , 1989 .

[20]  Xi-Ren Cao,et al.  Stochastic Learning and Optimization: A Sensitivity-Based Approach (International Series on Discrete Event Dynamic Systems) , 2007 .

[21]  James C. Spall,et al.  Introduction to Stochastic Search and Optimization. Estimation, Simulation, and Control (Spall, J.C. , 2007 .

[22]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[23]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[24]  Chun-Hung Chen,et al.  New development of optimal computing budget allocation for discrete event simulation , 1997, WSC '97.

[25]  Chun-Hung Chen,et al.  Optimal computing budget allocation for Monte Carlo simulation with application to product design , 2003, Simul. Model. Pract. Theory.

[26]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[27]  Fred W. Glover,et al.  Simulation optimization: a review, new developments, and applications , 2005, Proceedings of the Winter Simulation Conference, 2005..

[28]  Douglas C. Montgomery,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[29]  Chun-Hung Chen,et al.  Opportunity Cost and OCBA Selection Procedures in Ordinal Optimization for a Fixed Number of Alternative Systems , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[30]  Stephen E. Chick,et al.  New Two-Stage and Sequential Procedures for Selecting the Best Simulated System , 2001, Oper. Res..

[31]  U. Rieder,et al.  Markov Decision Processes , 2010 .

[32]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[33]  Chun-Hung Chen,et al.  Simulation Budget Allocation for Further Enhancing the Efficiency of Ordinal Optimization , 2000, Discret. Event Dyn. Syst..

[34]  Enver Yücesan,et al.  Discrete-event simulation optimization using ranking, selection, and multiple comparison procedures: A survey , 2003, TOMC.

[35]  A. Tamhane.  Design and Analysis of Experiments for Statistical Selection, Screening, and Multiple Comparisons , 1995 .

[36]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..