Pseudo-symmetries of Anosov maps and spectral statistics