Stability, convergence, and performance of an adaptive control algorithm applied to a randomly varying system

The stability and performance of a stochastic adaptive control algorithm applied to a randomly varying linear system is investigated. Using techniques from the theory of Markov chains, it is shown that loss functions on the input-output process converge to their expectation with respect to an invariant probability. It is shown that the convergence is geometric, establishing a form of stochastic exponential asymptotic stability for the closed-loop system. Further results include central limit theorems and the law of large numbers for the input-output and parameter processes and near consistency and optimality in the case where the disturbances are small.<<ETX>>

[1]  F. G. Foster.  On the Stochastic Matrices Associated with Certain Queuing Processes , 1953 .

[2]  K. Parthasarathy.  PROBABILITY MEASURES IN A METRIC SPACE , 1967 .

[3]  B. Mark.  On Self Tuning Regulators , 1972 .

[4]  R. Tweedie,et al.  Markov Chains with Continuous Components , 1979 .

[5]  P. Ramadge,et al.  Discrete Time Stochastic Adaptive Control , 1981 .

[6]  E. Nummelin,et al.  Geometric ergodicity of Harris recurrent Marcov chains with applications to renewal theory , 1982 .

[7]  R. L. Tweedie,et al.  Probability, Statistics and Analysis: Criteria for rates of convergence of Markov chains, with application to queueing and storage theory , 1983 .

[8]  E. Nummelin.  General irreducible Markov chains and non-negative operators: List of symbols and notation , 1984 .

[9]  P. Kumar,et al.  Adaptive control with the stochastic approximation algorithm: Geometry and convergence , 1985 .

[10]  P. Caines,et al.  A new approach to stochastic adaptive control , 1987, 1986 25th IEEE Conference on Decision and Control.

[11]  Petros A. Ioannou,et al.  Robust Discrete-Time Adaptive Control , 1986 .

[12]  Lei Guo,et al.  Asymptotically optimal adaptive control with consistent parameter estimates , 1987 .

[13]  Sean P. Meyn.  The Structural Robustness of Stochastic Systems , 1988 .

[14]  Guo Lei,et al.  A robust stochastic adaptive controller , 1988 .

[15]  P. Kumar,et al.  A robust adaptive minimum variance controller , 1989 .

[16]  S. Meyn.  Ergodic theorems for discrete time stochastic systems using a stochastic lyapunov function , 1989 .

[17]  Sean P. Meyn,et al.  Adaptive control for time‐varying systems: A combination of Martingale and Markov chain techniques , 1989 .

[18]  Lei Guo.  On adaptive stabilization of time-varying stochastic systems , 1990 .

[19]  Sean P. Meyn,et al.  Adaptive Control of Time Varying Stochastic Systems , 1990 .

[20]  Sean P. Meyn,et al.  Asymptotic behavior of stochastic systems possessing Markovian realizations , 1991 .