The effect of tangential drifts on neoclassical transport in stellarators close to omnigeneity