Vibrational stability of the rotating Roche model