The thermodynamic properties of Markov processes