Controlling atomic spin mixing via multiphoton transitions in a cavity