Quantum gates between mesoscopic spin ensembles