Spectral variation under congruence