On the resilience of scalar and vector vortex modes in turbulence