Uniformization for nonhomogeneous Markov chains

The discrete Poissonian representation for transition probabilities of homogeneous continuous-time Markov chains, known as uniformization or randomization, is extended to time-inhomogeneous chains. For computational purposes a discrete-time approximation is also provided.

[1]  Petar Todorovic,et al.  Markov Processes I , 1992 .

[2]  Micha Yadin,et al.  Randomization Procedures in the Computation of Cumulative-Time Distributions over Discrete State Markov Processes , 1984, Oper. Res..

[3]  A. Jensen,et al.  Markoff chains as an aid in the study of Markoff processes , 1953 .

[4]  Jürg Kohlas,et al.  Stochastic Methods of Operations Research , 1983 .

[5]  Ward Whitt,et al.  Some Useful Functions for Functional Limit Theorems , 1980, Math. Oper. Res..

[6]  Nico M. van Dijk,et al.  A Note on Error Bounds for Approximating Transition Probabilities in Continuous-time Markov Chains , 1988, Probability in the Engineering and Informational Sciences.

[7]  J. George Shanthikumar,et al.  Bounds and Approximations for the Transient Behavior of Continuous-Time Markov Chains , 1989, Probability in the Engineering and Informational Sciences.

[8]  Sheldon M. Ross,et al.  Approximating Transition Probabilities and Mean Occupation Times in Continuous-Time Markov Chains , 1987, Probability in the Engineering and Informational Sciences.

[9]  Valerie Isham,et al.  Non‐Negative Matrices and Markov Chains , 1983 .

[10]  Nico M. van Dijk,et al.  Truncation of Markov Chains with Applications to Queueing , 1991, Oper. Res..

[11]  Thomas G. Kurtz,et al.  Extensions of Trotter's operator semigroup approximation theorems , 1969 .

[12]  Kishor S. Trivedi,et al.  Numerical transient analysis of markov models , 1988, Comput. Oper. Res..

[13]  Nico M. van Dijk.  Approximate uniformization for continuous-time Markov chains with an application to performability analysis , 1988 .

[14]  Daniel Ocone.  Controlled Markov Processes: Time-Discretization (N. M. Van Dijk) , 1987 .

[15]  Terry Williams,et al.  Probability and Statistics with Reliability, Queueing and Computer Science Applications , 1983 .

[16]  Donald Gross,et al.  The Randomization Technique as a Modeling Tool and Solution Procedure for Transient Markov Processes , 1984, Oper. Res..

[17]  P. Billingsley,et al.  Convergence of Probability Measures , 1969 .

[18]  Theodor Meis,et al.  Numerical solution of partial differential equations , 1981 .

[19]  S. Ross.  A First Course in Probability , 1977 .

[20]  I. Gihman,et al.  Controlled Stochastic Processes , 1979 .

[21]  Iosif Ilitch Gikhman,et al.  Introduction to the theory of random processes , 1969 .

[22]  Philip Heidelberger,et al.  Sensitivity Analysis of Continuous Time Markov Chains Using Uniformization , 1987, Computer Performance and Reliability.

[23]  Kishor S. Trivedi.  Probability and Statistics with Reliability, Queuing, and Computer Science Applications , 1984 .

[24]  S. Karlin,et al.  A second course in stochastic processes , 1981 .