Exact non-Markovian dynamics of Gaussian quantum channels: Finite-time and asymptotic regimes