To evaluate the role of nonphonon energy relaxation mechanisms in quantum dots and in particular the role of electron-hole (e-h) interactions, we have studied femtosecond carrier dynamics in CdSe colloidal nanoparticles in which the e-h separation (coupling) is controlled using different types of surface ligands. In dots capped with hole accepting molecules, the e-h coupling is strongly reduced after the hole is transferred to a capping group. By re-exciting an electron within the conduction band at different stages of hole transfer and monitoring its relaxation back into the ground state, we observe a more than tenfold increase in the electron relaxation time (from 250 fs to 3 ps) after the completion of the hole transfer to the capping molecule. This strongly indicates that electron relaxation in quantum dots is dominated not by phonon emission but by the e-h energy transfer.