Stochastic Stability

This paper will survey the field of stochastic stability, with special emphasis on the "invariance" theorems and their potential application to systems with randomly varying coefficients. First, we will survey some of the basic ideas underlying the "stochastic Liapunov function" approach to stochastic stability, then the invariance theorems will be discussed in detail and, if time permits, an example given. In stability analyses of all types of deterministic dynamical systems, the concepts of co-limit set and invariant set play an important role. Let x,, t > 0, denote a bounded continuous solution to the detert — ministic ordinary differential equation x f(x), where f(-) is continuous. A point x is said to be in the CD-limit set if x. -> x for n some sequence t -»°°. A set B is said to be an invariant set if for each x e B, there is a path y , t e (-°°,°°) for which y e B and y = t t t , t e (-°°,°°). Thus, the invariant set contains entire trajectories. It is well known that the co-limit set is a closed non-empty invariant set. Also, V(-) be a Liapunov function with V(x) = -k(x) < 0, where k(') is continuous. Then by a very useful theorem of LaSalle, x. tends to the "C largest invariant set contained in {x = k(x) = 0} = K, as t ->«>. The theorem is important and useful, since in numerous applications the derivative k(-) of the Liapunov function V(-) is semi-definite, and the theorem gives a nice characterization of the subset of K to which x. tends. "C In fact, this characterization often enables us to determine the minimal set in K to which . x tends. O There are useful stochastic analogs to all the deterministic results (the dynamical system is a suitable dynamical system of measures), and they will be developed and explained. The concepts are felt to be useful for random parameter systems for the following reason. Suppose x = f(x,y), where the "parameter process" y, is a Markov process. Then (+^y+) will be Markov. Often, we are only interested in the asymptotic "C "C properties of x. . Indeed y, may even be stationary. Any stochastic "u t Liapunov function would have to take both x , y. into account in some ~c TJ way, and the "stochastic derivative" of that Liapunov function will often be semi-definite (it can't be a negative definite function of (x.,y ) t t if y, is stationary). The invariance theorems enter here to assist us in studying the asymptotic properties of the x, process. "C

[1]  S. Karlin,et al.  A second course in stochastic processes , 1981 .

[2]  Solomon Lefschetz,et al.  Stability by Liapunov's Direct Method With Applications , 1962 .

[3]  M. Freidlin,et al.  Random Perturbations of Dynamical Systems , 1984 .

[4]  J. P. Lasalle.  THE EXTENT OF ASYMPTOTIC STABILITY. , 1960, Proceedings of the National Academy of Sciences of the United States of America.

[5]  H. J. Kushner.  The concept of invariant set for stochastic dynamical systems, and applications to stochastic stability , 1967 .

[6]  Kevin S. McCurley,et al.  Ranking the web frontier , 2004, WWW '04.

[7]  J. Dai.  On Positive Harris Recurrence of Multiclass Queueing Networks: A Unified Approach Via Fluid Limit Models , 1995 .

[8]  Harold J. Kushner,et al.  Approximation and Weak Convergence Methods for Random Processes , 1984 .

[9]  P. Kumar,et al.  Theory and practice of recursive identification , 1985, IEEE Transactions on Automatic Control.

[10]  Hector Garcia-Molina,et al.  The Eigentrust algorithm for reputation management in P2P networks , 2003, WWW '03.

[11]  G. V. Anand.  Stability of Stochastic Systems , 1983 .

[12]  Eli Upfal,et al.  The Web as a graph , 2000, PODS.

[13]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[14]  Harold J. Kushner,et al.  On the stability of processes defined by stochastic difference-differential equations. , 1968 .

[15]  Gene H. Golub,et al.  Exploiting the Block Structure of the Web for Computing , 2003 .

[16]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.

[17]  R. Khasminskii.  Stochastic Stability of Differential Equations , 1980 .

[18]  Jack K. Hale,et al.  Sufficient conditions for stability and instability of autonomous functional- Differential equations. , 1965 .

[19]  W. Grassman.  Approximation and Weak Convergence Methods for Random Processes with Applications to Stochastic Systems Theory (Harold J. Kushner) , 1986 .

[20]  H. Young,et al.  The Evolution of Conventions , 1993 .

[21]  D. Sworder,et al.  Introduction to stochastic control , 1972 .

[22]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[23]  N. Krylov,et al.  Introduction to the Theory of Random Processes , 2002 .

[24]  H. Peyton Young,et al.  Individual Strategy and Social Structure , 2020 .

[25]  Carlos S. Kubrusly,et al.  Stochastic approximation algorithms and applications , 1973, CDC 1973.

[26]  Amy Nicole Langville,et al.  A Survey of Eigenvector Methods for Web Information Retrieval , 2005, SIAM Rev..