Non-Markovian dynamics under time-translation symmetry