A Quotient Construction on Markov Chains with Applications to the Theory of Generalized Simulated Annealing

In an earlier paper [14], we developed the first algorithm (to our knowledge) for computing the stochastically stable distribution of a perturbed Markov process. The primary tool was a novel quotient construction on Markov matrices. In this paper, we show that the ideas and techniques in that paper arise from a more fundamental construction on Markov chains, and have much wider applicability than simply to game theory (the application discussed in [14]). Besides leading to new results, our quotient construction leads to simpler proofs of known results and to simpler algorithms for known computations. In this paper, we present one example of the former—we give necessary and sufficient conditions for a Markov matrix to have a unique stable distribution—and one of the latter—we show that a variant of the algorithm in [14] can be used to compute the virtual energy levels of a generalized simulated annealing in a straightforward, recursive manner using basic matrix arithmetic.