Memory-based Stochastic Optimization

In this paper we introduce new algorithms for optimizing noisy plants in which each experiment is very expensive. The algorithms build a global non-linear model of the expected output at the same time as using Bayesian linear regression analysis of locally weighted polynomial models. The local model answers queries about confidence, noise, gradient and Hessians, and use them to make automated decisions similar to those made by a practitioner of Response Surface Methodology. The global and local models are combined naturally as a locally weighted regression. We examine the question of whether the global model can really help optimization, and we extend it to the case of time-varying functions. We compare the new algorithms with a highly tuned higher-order stochastic optimization algorithm on randomly-generated functions and a simulated manufacturing task. We note significant improvements in total regret, time to converge, and final solution quality.

[1]  M. Degroot.  Optimal Statistical Decisions , 1970 .

[2]  Harold J. Kushner,et al.  wchastic. approximation methods for constrained and unconstrained systems , 1978 .

[3]  Joseph E LeDoux,et al.  The brain and cognitive sciences , 1978, Annals of neurology.

[4]  Michel Installe,et al.  Stochastic approximation methods , 1978 .

[5]  Editors , 1986, Brain Research Bulletin.

[6]  George E. P. Box,et al.  Empirical Model‐Building and Response Surfaces , 1988 .

[7]  W. Cleveland,et al.  Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting , 1988 .

[8]  Christopher G. Atkeson,et al.  Using Local Models to Control Movement , 1989, NIPS.

[9]  Andrew W. Moore,et al.  Fast, Robust Adaptive Control by Learning only Forward Models , 1991, NIPS.

[10]  Christopher G. Atkeson,et al.  Memory-Based Learning Control , 1991, 1991 American Control Conference.

[11]  Russell Greiner,et al.  A Statistical Approach to Solving the EBL Utility Problem , 1992, AAAI.

[12]  Leslie Pack Kaelbling,et al.  Learning in embedded systems , 1993 .

[13]  Andrew W. Moore,et al.  Hoeffding Races: Accelerating Model Selection Search for Classification and Function Approximation , 1993, NIPS.

[14]  Gerald DeJong,et al.  Learning Search Control Knowledge for Deep Space Network Scheduling , 1993, ICML.

[15]  S. Botros.  Model-based techniques in motor learning and task optimization , 1994 .

[16]  Marcos Salganicoff,et al.  Active Exploration and Learning in real-Valued Spaces using Multi-Armed Bandit Allocation Indices , 1995, ICML.