Deviations from the Normal Time Regime of Single-File Diffusion