Diffusion as a damping mechanism for neutron-star oscillations