Importance sampling for Markov chains: computing variance and determining optimal measures

In this paper we describe several computational algorithms useful in studying importance sampling (IS) for Markov chains. Our algorithms compute optimal IS measures and evaluate the estimate variance for a given measure. As knowledge of the optimal IS measure implies knowledge of the quantity to be estimated, our algorithms produce this quantity as a by-product. Since effective IS measures must often closely approximate the optimal measure, the use of these algorithms for small problems may produce in sights that lead to effective measures for larger problems of actual interest. We consider two classes of problems: hitting times and fixed-horizon costs.