Searching for links between magnetic fields and stellar evolution: II. The evolution of magnetic fie