Constraining the growth factor with baryon oscillations