Unravelling cosmological perturbations