Pairing versus phase coherence of doped holes in distinct quantum spin backgrounds