On Generalizing Beyond Domains in Cross-Domain Continual Learning