Lumping Markov Chains with Silent Steps

A silent step in a dynamic system is a step that is considered unobservable and that can be eliminated. We define a Markov chain with silent steps as a class of Markov chains parameterized with a special real number tau. When tau goes to infinity silent steps become immediate, i.e. timeless, and therefore unobservable. To facilitate the elimination of these steps while preserving performance measures, we introduce a notion of lumping for the new setting. To justify the lumping we first extend the standard notion of ordinary lumping to the setting of discontinuous Markov chains, processes that can do infinitely many transitions in finite time. Then, we give a direct connection between the two lumpings for the case when tau is infinite. The results of this paper can serve as a correctness criterion and a method for the elimination of silent (tau) steps in Markovian process algebras

[1]  P. Yu. Chebotarev,et al.  On Determining the Eigenprojection and Components of a Matrix , 2002 .

[2]  Pedro R. D'Argenio,et al.  Algebras and Automata for Timed and Stochastic Systems , 1999 .

[3]  Mario Bravetti,et al.  Real Time and Stochastic Time , 2004, SFM.

[4]  Kishor S. Trivedi,et al.  An Aggregation Technique for the Transient Analysis of Stiff Markov Chains , 1986, IEEE Transactions on Computers.

[5]  Joost-Pieter Katoen,et al.  General Distributions in Process Algebra , 2002, European Educational Forum: School on Formal Methods and Performance Analysis.

[6]  Mario Bravetti,et al.  Tutte le Algebre Insieme: Concepts, Discussions and Relations of Stochastic Process Algebras with General Distributions , 2004, Validation of Stochastic Systems.

[7]  S. Campbell.  Singular Systems of Differential Equations , 1980 .

[8]  R. G. Cooke.  Functional Analysis and Semi-Groups , 1949, Nature.

[9]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[10]  Trung Dinh Tran,et al.  Semistable Operators and Singularly Perturbed Differential Equations , 1999 .

[11]  Kishor S. Trivedi,et al.  On the Solution of GSPN Reward Models , 1991, Perform. Evaluation.

[12]  S. Sastry,et al.  Hierarchical aggregation of singularly perturbed finite state Markov processes , 1983 .

[13]  François Delebecque,et al.  Optimal control of markov chains admitting strong and weak interactions , 1981, Autom..

[14]  P. Buchholz.  Exact and ordinary lumpability in finite Markov chains , 1994, Journal of Applied Probability.

[15]  B. Nordstrom.  FINITE MARKOV CHAINS , 2005 .

[16]  Holger Hermanns,et al.  Interactive Markov Chains , 2002, Lecture Notes in Computer Science.

[17]  Jane Hillston,et al.  A compositional approach to performance modelling , 1996 .

[18]  F. Delebecque.  A Reduction Process for Perturbed Markov Chains , 1983 .

[19]  Winfried K. Grassmann,et al.  Immediate Events in Markov Chains , 1995 .