(Non-)Universality in symmetric quantum circuits: Why Abelian symmetries are special