Averages of the number of points on elliptic curves