Evidence for charge drift modulation at intermediate solar activity from the flux variation of protons and α particles