Accelerating Transformers with Spectrum-Preserving Token Merging