Theory of spin inertia in singly charged quantum dots