Simultaneous perturbation stochastic approximation of nonsmooth functions

Abstract A simultaneous perturbation stochastic approximation (SPSA) method has been developed in this paper, using the operators of perturbation with the Lipschitz density function. This model enables us to use the approximation of the objective function by twice differentiable functions and to present their gradients by volume integrals. The calculus of the stochastic gradient by means of this presentation and likelihood ratios method is proposed, that can be applied to create SPSA algorithms for a wide class of perturbation densities. The convergence of the SPSA algorithms is proved for Lipschitz objective functions under quite general conditions. The rate of convergence O 1 k γ , 1 γ 2 of the developed algorithms has been established for functions with a sharp minimum, as well as the dependence of the rate of convergence is explored theoretically as well as by computer simulation. The applicability of the presented algorithm is demonstrated by applying it to minimization of the mean absolute pricing error for the calibration of the Heston stochastic volatility model.

[1]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[2]  Jason H. Goodfriend,et al.  Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method , 1995 .

[3]  U. Neri.  Distributions and Fourier transforms , 1971 .

[4]  Stefan Friedrich,et al.  Topology , 2019, Arch. Formal Proofs.

[5]  A. M. Gupal,et al.  Algorithm for the minimization of discontinuous functions , 1977 .

[6]  J. Blum.  Multidimensional Stochastic Approximation Methods , 1954 .

[7]  R. Rockafellar.  Directionally Lipschitzian Functions and Subdifferential Calculus , 1979 .

[8]  J. Dieudonne.  Foundations of Modern Analysis , 1969 .

[9]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[10]  M. T. Wasan.  Stochastic Approximation , 1969 .

[11]  Leonidas Sakalauskas,et al.  Nonlinear stochastic programming by Monte-Carlo estimators , 2002, Eur. J. Oper. Res..

[12]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[13]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[14]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[15]  Y. Ermoliev,et al.  The Minimization of Semicontinuous Functions: Mollifier Subgradients , 1995 .

[16]  F. Downton.  Stochastic Approximation , 1969, Nature.

[17]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[18]  Charles Leake,et al.  Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method , 1994 .

[19]  S. Heston.  A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options , 1993 .