Semiconductor quantum dots reveal dipolar coupling from exciton to ligand vibration