Taylor series expansions for stationary Markov chains