On the consistency of (partially-)massless matter couplings in de Sitter space