Stochastic controllability and stochastic Lyapunov functions with applications to adaptive and nonlinear systems

Sufficient conditions are established under which the law of large numbers and related ergodic theorems hold for nonlinear stochastic systems operating under feedback. It is shown that these conditions hold whenever a moment condition is satisfied, which may be interpreted as a generalization of the martingale property.

[1]  F. G. Foster.  On the Stochastic Matrices Associated with Certain Queuing Processes , 1953 .

[2]  J. Doob.  Stochastic processes , 1953 .

[3]  R. E. Kalman,et al.  Control System Analysis and Design Via the “Second Method” of Lyapunov: I—Continuous-Time Systems , 1960 .

[4]  G. Zames.  On the input-output stability of time-varying nonlinear feedback systems--Part II: Conditions involving circles in the frequency plane and sector nonlinearities , 1966 .

[5]  G. Zames.  On the input-output stability of time-varying nonlinear feedback systems Part one: Conditions derived using concepts of loop gain, conicity, and positivity , 1966 .

[6]  Finite regular invariant measures for Feller processes , 1968 .

[7]  P. Billingsley,et al.  Convergence of Probability Measures , 1969 .

[8]  S. Foguel.  POSITIVE OPERATORS ON C(X) , 1969 .

[9]  G. P. Szegö,et al.  Stability theory of dynamical systems , 1970 .

[10]  M. Rosenblatt.  Markov Processes, Structure and Asymptotic Behavior , 1971 .

[11]  Vasile Mihai Popov,et al.  Hyperstability of Control Systems , 1973 .

[12]  R. Cogburn.  A Uniform Theory for Sums of Markov Chain Transition Probabilities , 1975 .

[13]  James R. Brown,et al.  Ergodic theory and topological dynamics , 1976 .

[14]  V. Solo.  Time series recursions and stochastic approximation , 1979, Bulletin of the Australian Mathematical Society.

[15]  R. Tweedie,et al.  Markov Chains with Continuous Components , 1979 .

[16]  Michael G. Safonov,et al.  Stability and Robustness of Multivariable Feedback Systems , 1980 .

[17]  P. Ramadge,et al.  Discrete Time Stochastic Adaptive Control , 1981 .

[18]  E. Nummelin,et al.  Geometric ergodicity of Harris recurrent Marcov chains with applications to renewal theory , 1982 .

[19]  Eduardo Sontag.  A Lyapunov-Like Characterization of Asymptotic Controllability , 1983, SIAM Journal on Control and Optimization.

[20]  R. L. Tweedie,et al.  Probability, Statistics and Analysis: Criteria for rates of convergence of Markov chains, with application to queueing and storage theory , 1983 .

[21]  J. F. C. Kingman,et al.  Probability, statistics, and analysis , 1983 .

[22]  E. Nummelin.  General irreducible Markov chains and non-negative operators: List of symbols and notation , 1984 .

[23]  E. Nummelin.  General irreducible Markov chains and non-negative operators: Preface , 1984 .

[24]  P. Caines,et al.  A new approach to stochastic adaptive control , 1987, 1986 25th IEEE Conference on Decision and Control.

[25]  P. Caines.  Linear Stochastic Systems , 1988 .

[26]  S. Meyn.  Ergodic theorems for discrete time stochastic systems using a stochastic lyapunov function , 1989 .

[27]  Sean P. Meyn,et al.  Adaptive control for time‐varying systems: A combination of Martingale and Markov chain techniques , 1989 .

[28]  Eduardo Sontag,et al.  Controllability of Nonlinear Discrete-Time Systems: A Lie-Algebraic Approach , 1990, SIAM Journal on Control and Optimization.