A SUCCESSIVE LUMPING PROCEDURE FOR A CLASS OF MARKOV CHAINS

A class of Markov chains we call successively lumbaple is specified for which it is shown that the stationary probabilities can be obtained by successively computing the stationary probabilities of a propitiously constructed sequence of Markov chains. Each of the latter chains has a(typically much) smaller state space and this yields significant computational improvements. We discuss how the results for discrete time Markov chains extend to semi-Markov processes and continuous time Markov processes. Finally, we will study applications of successively lumbaple Markov chains to classical reliability and queueing models.

[1]  Marcel F. Neuts,et al.  Matrix-Geometric Solutions in Stochastic Models , 1981 .

[2]  Rhonda Righter.  Optimal policies for scheduling repairs and allocating heterogeneous servers , 1996 .

[3]  Michael N. Katehakis,et al.  Optimal Repair Allocation in a Series System , 1984, Math. Oper. Res..

[4]  V. Pan,et al.  Methods of aggregation , 1980 .

[5]  Michael N. Katehakis,et al.  On the maintenance of systems composed of highly reliable components , 1989 .

[6]  S. M. Ross,et al.  On the optimal assignment of servers and a repairman , 1980, Journal of Applied Probability.

[7]  V. B. Yap,et al.  Similar States in Continuous-Time Markov Chains , 2009, Journal of Applied Probability.

[8]  Michael N. Katehakis,et al.  On the optimal maintenance of systems and control of arrivals in queues , 1995 .

[9]  Qi Zhang,et al.  Performance-Guided Load (Un)balancing under Autocorrelated Flows , 2008, IEEE Transactions on Parallel and Distributed Systems.

[10]  John G. Kemeny,et al.  Finite Markov Chains. , 1960 .

[11]  Esther Frostig.  Jointly optimal allocation of a repairman and optimal control of service rate for machine repairman problem , 1999, Eur. J. Oper. Res..

[12]  B. Nordstrom.  FINITE MARKOV CHAINS , 2005 .

[13]  Samuel S. Chiu,et al.  A Method to Calculate Steady-State Distributions of Large Markov Chains by Aggregating States , 1987, Oper. Res..

[14]  Victoria Ungureanu,et al.  Deferred Assignment Scheduling in Cluster-Based Servers , 2006, Cluster Computing.

[15]  Ger Koole,et al.  ON DEVIATION MATRICES FOR BIRTH–DEATH PROCESSES , 2001, Probability in the Engineering and Informational Sciences.

[16]  Paul J. Schweitzer,et al.  Iterative Aggregation-Disaggregation Procedures for Discounted Semi-Markov Reward Processes , 1985, Oper. Res..

[17]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[18]  Ger Koole,et al.  On the convergence of the power series algorithm , 2000, Perform. Evaluation.