Arithmetic group symmetry and finiteness properties of Torelli groups