Off-resonant manipulation of spins in diamond via precessing magnetization of a proximal ferromagnet