Manipulating photonic spin accumulation with a magnetic field