Hysteresis and instabilities in a spheroid in precession near the resonance with the tilt-over mode