Congruential automorphism groups of general matrices