The cross-correlation of galaxies in absorption with the Lyman $\alpha$ forest