Schr"odinger invariance and space-time symmetries