High-order inertial phase shifts for time-domain atom interferometers