Extra invariance and Balian-Low type obstructions for Gabor spaces