The variance in stochastic transport problems with Markovian mixing