Deformations of compact Prym fibrations to Hitchin systems