Vector and tensor contributions to the curvature perturbation at second order