Aggregation and time scale analysis of perturbed Markov systems

Abstract : Analysis of systems with many time scales is important in many engineering applications. This thesis addresses the approximation and decomposition of Markov processes which exhibit such multiple time scales. An algorithm is presented for the decomposition of explicitly perturbed, finite state, continuous time Markov processes. An approximation of the probability transition function which converges uniformly to zero over T greater than or equal to O is obtained. The algorithm extends previous work by providing a straightforward algorithm which has a direct probabilistic interpretation, particularly with respect to the role played by transient states. This result is then extended to consider semi Markov and discrete time Markov processes as well. Decomposition of perturbed positive systems is also addressed. Finally, the Markov process decomposition algorithm is expressed in graphical terms and applied to a problem of determining the multiple time scale structure of a fault-tolerant system model.