Instabilities of spin-liquid states in a quantum kagome antiferromagnet