Transient Solutions for Markov Chains

Much of the theory developed for solving Markov chain models is devoted to obtaining steady state measures, that is, measures for which the observation interval (0, t) is “sufficiently large” (t → ∞). These measures are indeed approximations of the behavior of the system for a finite, but long, time interval, where long means with respect to the interval of time between occurrences of events in the system. However, an increasing number of applications requires the calculation of measures during a relatively “short” period of time. These are the so-called transient measures. In these cases the steady state measures are not good approximations for the transient, and one has to resort to different techniques to obtain the desired quantities.

[1]  A. Jensen,et al.  Markoff chains as an aid in the study of Markoff processes , 1953 .

[2]  Prem S. Puri,et al.  A method for studying the integral functionals of stochastic processes with applications: I. Markov chain case , 1971, Journal of Applied Probability.

[3]  H. Weisberg,et al.  The Distribution of Linear Combinations of Order Statistics from the Uniform Distribution , 1971 .

[4]  Prem S. Puri,et al.  A method for studying the integral functional of stochastic processes with applications , 1972 .

[5]  Erhan Çinlar,et al.  Introduction to stochastic processes , 1974 .

[6]  W. Grassmann.  Transient solutions in Markovian queues : An algorithm for finding them and determining their waiting-time distributions , 1977 .

[7]  Winfried K. Grassmann.  Transient solutions in markovian queueing systems , 1977, Comput. Oper. Res..

[8]  C. Loan,et al.  Nineteen Dubious Ways to Compute the Exponential of a Matrix , 1978 .

[9]  John F. Meyer,et al.  On Evaluating the Performability of Degradable Computing Systems , 1980, IEEE Transactions on Computers.

[10]  John F. Meyer,et al.  Performability Evaluation of the SIFT Computer , 1980, IEEE Transactions on Computers.

[11]  Kishor S. Trivedi.  Probability and Statistics with Reliability, Queuing, and Computer Science Applications , 1984 .

[12]  W. Grassmann.  The GI/PH/1 Queue: a Method to Find The Transition Matrix , 1982 .

[13]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[14]  John F. Meyer,et al.  Closed-Form Solutions of Performability , 1982, IEEE Transactions on Computers.

[15]  Gene H. Golub,et al.  Matrix computations , 1983 .

[16]  Terry Williams,et al.  Probability and Statistics with Reliability, Queueing and Computer Science Applications , 1983 .

[17]  T. Matsunawa.  The exact and approximate distributions of linear combinations of selected order statistics from a uniform distribution , 1985 .

[18]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[19]  Kishor S. Trivedi,et al.  An Aggregation Technique for the Transient Analysis of Stiff Markov Chains , 1986, IEEE Transactions on Computers.

[20]  Marco Ajmone Marsan,et al.  On Petri nets with deterministic and exponentially distributed firing times , 1986, European Workshop on Applications and Theory of Petri Nets.

[21]  Edmundo de Souza e Silva,et al.  Calculating Cumulative Operational Time Distributions of Repairable Computer Systems , 1986, IEEE Transactions on Computers.

[22]  Harmen R. van As.  Transient Analysis of Markovian Queueing Systems and Its Application to Congestion-Control Modeling , 1986, IEEE J. Sel. Areas Commun..

[23]  Philip Heidelberger,et al.  Analysis of Performability for Stochastic Models of Fault-Tolerant Systems , 1986, IEEE Transactions on Computers.

[24]  Philip Heidelberger,et al.  Sensitivity Analysis of Continuous Time Markov Chains Using Uniformization , 1987, Computer Performance and Reliability.

[25]  Winfried K. Grassmann,et al.  Means and variances of time averages in Markovian environments , 1987 .

[26]  Sheldon M. Ross,et al.  Approximating Transition Probabilities and Mean Occupation Times in Continuous-Time Markov Chains , 1987, Probability in the Engineering and Informational Sciences.

[27]  Nico M. van Dijk.  Approximate uniformization for continuous-time Markov chains with an application to performability analysis , 1988 .

[28]  Miroslaw Malek,et al.  Survey of software tools for evaluating reliability, availability, and serviceability , 1988, CSUR.

[29]  Kishor S. Trivedi,et al.  Analysis of Stiff Markov Chains , 1989, INFORMS J. Comput..

[30]  Kishor S. Trivedi,et al.  SPNP: stochastic Petri net package , 1989, Proceedings of the Third International Workshop on Petri Nets and Performance Models, PNPM89.

[31]  Edmundo de Souza e Silva,et al.  Calculating availability and performability measures of repairable computer systems using randomization , 1989, JACM.

[32]  Ward Whitt,et al.  Calculating time-dependent performance measures for the M/M/1 queue , 1989, IEEE Trans. Commun..

[33]  J. George Shanthikumar,et al.  Bounds and Approximations for the Transient Behavior of Continuous-Time Markov Chains , 1989, Probability in the Engineering and Informational Sciences.

[34]  Nico M. Van Dijk,et al.  The Importance of Bias Terms for Error Bounds and Comparison Results , 1989 .

[35]  Krishna R. Pattipati,et al.  On the Computational Aspects of Performability Models of Fault-Tolerant Computer Systems , 1990, IEEE Trans. Computers.

[36]  Edmundo de Souza e Silva,et al.  Analyzing Scheduled Maintenance Policies for Repairable Computer Systems , 1990, IEEE Trans. Computers.

[37]  Kishor S. Trivedi,et al.  NUMERICAL EVALUATION OF PERFORMABILITY AND JOB COMPLETION TIME IN REPAIRABLE FAULT-TOLERANT SYSTEMS. , 1990 .

[38]  L. Donatiello,et al.  On Evaluating the Cumulative Performance Distribution of Fault-Tolerant Computer Systems , 1991, IEEE Trans. Computers.

[39]  Christoph Lindemann,et al.  An improved numerical algorithm for calculating steady-state solutions of deterministic and stochastic Petri net models , 1991, Proceedings of the Fourth International Workshop on Petri Nets and Performance Models PNPM91.

[40]  Hany H. Ammar,et al.  Performability Analysis of Distributed Real-Time Systems , 1991, IEEE Trans. Computers.

[41]  John F. Meyer,et al.  Performability: A Retrospective and Some Pointers to the Future , 1992, Perform. Evaluation.

[42]  N. M. van Dijk,et al.  Uniformization for nonhomogeneous Markov chains , 1992, Oper. Res. Lett..

[43]  Edmundo de Souza e Silva,et al.  Performability Analysis of Computer Systems: From Model Spacification to Solution , 1992, Perform. Evaluation.

[44]  Y. Saad.  Analysis of some Krylov subspace approximations to the matrix exponential operator , 1992 .

[45]  Kishor S. Trivedi,et al.  Multiprocessor performability analysis , 1993 .

[46]  Gerardo Rubino,et al.  Interval availability distribution computation , 1993, FTCS-23 The Twenty-Third International Symposium on Fault-Tolerant Computing.

[47]  Raymond A. Marie,et al.  The uniformized power method for transient solutions of Markov processes , 1993, Comput. Oper. Res..

[48]  Winfried K. Grassmann.  Means and Variances in Markov Reward Systems , 1993 .

[49]  Kishor S. Trivedi,et al.  Automated Generation and Analysis of Markov Reward Models Using Stochastic Reward Nets , 1993 .

[50]  Gerardo Rubino,et al.  Transient analysis of the M/M/1 queue , 1993, Advances in Applied Probability.

[51]  Krishna R. Pattipati,et al.  A Unified Framework for the Performability Evaluation of Fault-Tolerant Computer Systems , 1993, IEEE Trans. Computers.

[52]  William H. Sanders,et al.  Performability evaluation of CSMA/CD and CSMA/DCR protocols under transient fault conditions , 1993 .

[53]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[54]  Hoon Choi,et al.  Markov Regenerative Stochastic Petri Nets , 1994, Perform. Evaluation.

[55]  W. Whitt,et al.  Multidimensional Transform Inversion with Applications to the Transient M/G/1 Queue , 1994 .

[56]  W. Whitt,et al.  The transient BMAP/G/l queue , 1994 .

[57]  William H. Sanders,et al.  Reward Model Solution Methods with Impulse and Rate Rewards: An Algorithm and Numerical Results , 1994, Perform. Evaluation.

[58]  Y. Takahashi,et al.  Transient Analysis of Fluid Model for ATM Statistical Multiplexer , 1995, Perform. Evaluation.

[59]  Edmundo de Souza e Silva,et al.  Calculating transient distributions of cumulative reward , 1995, SIGMETRICS '95/PERFORMANCE '95.

[60]  Richard R. Muntz,et al.  Polling systems with server timeouts and their application to token passing networks , 1995, TNET.

[61]  Christoph Lindemann,et al.  DSPNexpress: A Software Package for the Efficient Solution of Deterministic and Stochastic Petri Nets , 1993, Perform. Evaluation.

[62]  Juan A. Carrasco,et al.  Regenerative randomization: theory and application examples , 1995, SIGMETRICS '95/PERFORMANCE '95.

[63]  Richard R. Muntz,et al.  Efficient Solutions for a Class of Non-Markovian Models , 1995 .

[64]  Hisashi Kobayashi,et al.  Transient Solutions for the Buffer Behavior in Statistical Multiplexing , 1995, Perform. Evaluation.

[65]  M.A. Qureshi,et al.  The UltraSAN Modeling Environment , 1995, Perform. Evaluation.

[66]  John F. Meyer.  Performability evaluation: where it is and what lies ahead , 1995, Proceedings of 1995 IEEE International Computer Performance and Dependability Symposium.

[67]  B. Philippe,et al.  Transient Solutions of Markov Processes by Krylov Subspaces , 1995 .

[68]  Bruno Sericola,et al.  Performability Analysis: A New Algorithm , 1996, IEEE Trans. Computers.

[69]  John F. Meyer,et al.  Performability management in distributed database systems: an adaptive concurrency control protocol , 1996, Proceedings of MASCOTS '96 - 4th International Workshop on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[70]  Kishor S. Trivedi,et al.  THE SYSTEM AVAILABILITY ESTIMATOR , 1996 .

[71]  Bruno Sericola.  Availability Analysis and Stationary Regime Detection of Markov Processes , 1996 .

[72]  Raymond A. Marie,et al.  Efficient Solutions for an Approximation Technique for the Transient Analysis of Markovian Models , 1996 .

[73]  Richard R. Muntz,et al.  Gated time-limited polling systems , 1997, PMCCN.

[74]  Richard R. Muntz,et al.  Performance/Availability Modeling with te TANGRAM-II Modeling Environment , 1998, Perform. Evaluation.

[75]  E. D. S. E. Silva,et al.  An algorithm to calculate transient distributions of cumulative rate and impulse based reward , 1998 .