Mixing properties of the Swendsen-Wang process on classes of graphs