Change-point monitoring for online stochastic approximations

We consider stochastic approximations in a quickly changing non-stationary environment. We assume the parameters of the system are subject to sudden discontinuous changes, which we refer to as regime-switching. We are interested in problems characterized by frequent significant jumps with no a priori knowledge about the regimes. Our approach is based on constant step size stochastic approximation. While larger step sizes have the advantage of fast adaptation, smaller step sizes provide more precise estimates of the target value once the process is close to stationary. We propose to use a small constant step size combined with change-point monitoring, and to reset the process at a value closer to the new target when a change is detected. Stochastic approximation and change-point monitoring complement each other by achieving high precision as well as cutting down the convergence time. We give a theoretical characterization and discuss the tradeoff between precision and fast adaptation. We also introduce a new monitoring scheme, the regression-based hypothesis test, which performs comparably well to Page-Hinkley's test and the CUSUM of residuals. The novelty of our approach is (a) the combination of change-point monitoring to stochastic approximation in a regime-switching environment and (b) the introduction of a new monitoring scheme. We provide an asymptotic analysis of this method and we show weak convergence to a limiting switching ODE for the non-reset method, and to a hybrid DE for a reset method that we propose.