Constraining Asymmetric Dark Matter using colliders and direct detection