Preserving Cross-Linguality of Pre-trained Models via Continual Learning