Flow fluctuations from early-time correlations in nuclear collisions