How many matrices can be spectrally balanced simultaneously?