Non-Markovian dynamics of charge carriers in quantum dots