Geometric U-folds in four dimensions