Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach

Prologue and Preliminaries: Introduction and overview- Mathematical preliminaries. Markovian models.- Singularly perturbed Markov chains: Asymptotic expansion: Irreducible generators. Asymptotic normality and exponential bounds. Asymptotic expansion: Weak and strong interactions. Weak and strong interactions: Asymptotic properties and ramification.- Optimizations and numerical methods: Markov decision problems. Stochastic control of dynamical systems. Numerical methods for control and optimization.