The size of semigroup orbits modulo primes