Effects of cold dark matter decoupling and pair annihilation on cosmological perturbations