It is known that some problems of almost sure convergence for stochastic approximation processes can be analyzed via an ordinary differential equation (ODE) obtained by suitable averaging. The goal of this paper is to show that the asymptotic behavior of such a process can be related to the asymptotic behavior of the ODE without any particular assumption concerning the dynamics of this ODE. The main results are as follows: a) The limit sets of trajectory solutions to the stochastic approximation recursion are, under classical assumptions, almost surely nonempty compact connected sets invariant under the flow of the ODE and contained in its set of chain-recurrence. b) If the gain parameter goes to zero at a suitable rate depending on the expansion rate of the ODE, any trajectory solution to the recursion is almost surely asymptotic to a forward trajectory solution to the ODE.
[1]
J. Neveu.
Bases mathématiques du calcul des probabilités
,
1966
.
[2]
C. Conley.
Isolated Invariant Sets and the Morse Index
,
1978
.
[3]
E. Eweda,et al.
Quadratic mean and almost-sure convergence of unbounded stochastic approximation algorithms with correlated observations
,
1983
.
[4]
T Poggio,et al.
Regularization Algorithms for Learning That Are Equivalent to Multilayer Networks
,
1990,
Science.
[5]
R. Pemantle,et al.
Nonconvergence to Unstable Points in Urn Models and Stochastic Approximations
,
1990
.
[6]
Halbert White,et al.
Artificial Neural Networks: Approximation and Learning Theory
,
1992
.
[7]
Asymptotic phase, shadowing and reaction-diffusion systems
,
1993
.