How to Combine Fast Heuristic Markov Chain Monte Carlo with Slow Exact Sampling