Polarization memory in single quantum dots