Transient Solution of Acyclic Markov Chains

Continuous time Markov chains are commonly used in system reliability modeling. Increasing system complexity and non-Markovian behavior can drastically increase the size of a Markov model''s state space. Special approximation techniques and numerical methods have been introduced to reduce the resources needed to solve Markov chain models. In this paper we discuss a method for automatically deriving exact transient solutions of Markov chains. The solutions derived are symbolic in $t$. Our approach can also provide solutions that are symbolic in other parameters. We extend our method to include parametric sensitivity analysis of the transient solution, and to provide cumulative measures of Markov chain behavior. We present three examples, one to show the use of our method in evaluating approximate solution techniques, one showing parametric sensitivity analysis of a large Markov model, and one demonstrating the computation of cumulative measures for an acyclic Markov reward processes.