Smoothing methods for variance reduction in simulation of Markov chains

This paper is concerned with the problem of applying simulation to efficiently estimate the long-run average cost m associated with a Markov chain {Xn } when a cost F(Xn – 1, Xn ) is incurred during the nth state transition, n = 1,2, . . . Our approach is to replace the cost structure F by a (smoother) cost structure F’ which provably results in the same long-run average cost m, and (hopefully) results in easier simulation. We show that the smoothing techniques proposed in this paper lead to variance reduction when applied to two state Markov chains, and we also present empirical results that show that the application of these techniques can lead to variance reduction on more general Markov chains.