Optimal group size is unstable