Adaptive stochastic approximation by the simultaneous perturbation method

Stochastic approximation (SA) has long been applied for problems of minimizing loss functions or root-finding with noisy input information. As with all stochastic search algorithms, there are adjustable algorithm coefficients that must be specified and that can have a profound effect on algorithm performance. It is known that picking these coefficients according to an SA analogue of the deterministic Newton-Raphson algorithm provides an optimal or near-optimal form of the algorithm. This paper presents a general adaptive SA algorithm that is based on an easy method for estimating the Hessian matrix at each iteration while concurrently estimating the primary parameters of interest. The approach applies in both the gradient-free optimization (Kiefer-Wolfowitz) and root-finding/stochastic gradient-based (Robbins-Monro) settings and is based on the "simultaneous perturbation" idea introduced previously.

[1]  Ronald R. Luman,et al.  Upgrading Complex Systems of Systems: A CAIV Methodology for Warfare Area Requirements Allocation , 2000 .

[2]  Pierre L'Ecuyer,et al.  Global Stochastic Optimization with Low-Dispersion Point Sets , 1998, Oper. Res..

[3]  László Gerencsér,et al.  Convergence rate of moments in stochastic approximation with simultaneous perturbation gradient approximation and resetting , 1999, IEEE Trans. Autom. Control..

[4]  A. V. Vande Wouwer,et al.  On the use of simultaneous perturbation stochastic approximation for neural network training , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[5]  J. Spall,et al.  Model-free control of nonlinear stochastic systems with discrete-time measurements , 1998, IEEE Trans. Autom. Control..

[6]  J. Spall.  Implementation of the simultaneous perturbation algorithm for stochastic optimization , 1998 .

[7]  J. L. Maryak.  Some guidelines for using iterate averaging in stochastic approximation , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[8]  J. Spall.  Accelerated second-order stochastic optimization using only function measurements , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[9]  Rui J. P. de Figueiredo,et al.  Learning rules for neuro-controller via simultaneous perturbation , 1997, IEEE Trans. Neural Networks.

[10]  J. Dippon,et al.  Weighted Means in Stochastic Approximation of Minima , 1997 .

[11]  J. Spall,et al.  Optimal random perturbations for stochastic approximation using a simultaneous perturbation gradient approximation , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[12]  Payman Sadegh,et al.  Constrained optimization via stochastic approximation with a simultaneous perturbation gradient approximation , 1997, Autom..

[13]  C. Kao,et al.  A modified quasi-newton method for optimization in simulation , 1997 .

[14]  D. C. Chin,et al.  Comparative study of stochastic algorithms for system optimization based on gradient approximations , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[15]  James C. Spall,et al.  A one-measurement form of simultaneous perturbation stochastic approximation , 1997, Autom..

[16]  Han-Fu Chen,et al.  A Stochastic Approximation Algorithm with Random Differences , 1996 .

[17]  R. Brennan,et al.  Stochastic optimization applied to a manufacturing system operation problem , 1995, Winter Simulation Conference Proceedings, 1995..

[18]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[19]  D. C. Chin,et al.  A more efficient global optimization algorithm based on Styblinski and Tang , 1994, Neural Networks.

[20]  Boris Polyak,et al.  Acceleration of stochastic approximation by averaging , 1992 .

[21]  G. Pflug,et al.  Stochastic approximation and optimization of random systems , 1992 .

[22]  G. Yin,et al.  Averaging procedures in adaptive filtering: an efficient approach , 1992 .

[23]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[24]  Lennart Ljung,et al.  Applications to adaptation algorithms , 1992 .

[25]  Harro Walk.  Foundations of stochastic approximation , 1992 .

[26]  G. Pflug.  Applicational aspects of stochastic approximation , 1992 .

[27]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[28]  M. A. Styblinski,et al.  Experiments in nonconvex optimization: Stochastic approximation with function smoothing and simulated annealing , 1990, Neural Networks.

[29]  J. Spall.  A stochastic approximation algorithm for large-dimensional systems in the Kiefer-Wolfowitz setting , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[30]  C. Z. Wei.  Multivariate Adaptive Stochastic Approximation , 1987 .

[31]  S. Evans,et al.  On the almost sure convergence of a general stochastic approximation procedure , 1986, Bulletin of the Australian Mathematical Society.

[32]  D. Ruppert.  A Newton-Raphson Version of the Multivariate Robbins-Monro Procedure , 1985 .

[33]  M. Metivier,et al.  Applications of a Kushner and Clark lemma to general classes of stochastic algorithms , 1984, IEEE Trans. Inf. Theory.

[34]  E. Eweda,et al.  Second-order convergence analysis of stochastic adaptive linear filtering , 1983 .

[35]  V. Nollau.  Kushner, H. J./Clark, D. S., Stochastic Approximation Methods for Constrained and Unconstrained Systems. (Applied Mathematical Sciences 26). Berlin‐Heidelberg‐New York, Springer‐Verlag 1978. X, 261 S., 4 Abb., DM 26,40. US $ 13.20 , 1980 .

[36]  R. Laha.  Probability Theory , 1979 .

[37]  V. Fabian.  On Asymptotically Efficient Recursive Estimation , 1978 .

[38]  Mikhail Borisovich Nevelʹson,et al.  Stochastic Approximation and Recursive Estimation , 1976 .

[39]  Carlos S. Kubrusly,et al.  Stochastic approximation algorithms and applications , 1973, CDC 1973.

[40]  F. Downton.  Stochastic Approximation , 1969, Nature.

[41]  M. T. Wasan.  Stochastic Approximation , 1969 .

[42]  V. Fabian.  On Asymptotic Normality in Stochastic Approximation , 1968 .

[43]  J. H. Venter.  An extension of the Robbins-Monro procedure , 1967 .

[44]  J. Blum.  Approximation Methods which Converge with Probability one , 1954 .