Silicon is promising for spin-based quantum computation because nuclear spins, a source of magnetic noise, may be eliminated through isotopic enrichment. Long spin decoherence times T2 have been measured in isotope-enriched silicon but come far short of the T2=2T1 limit. The effect of nuclear spins on T2 is well established. However, the effect of background electron spins from ever present residual phosphorus impurities in silicon can also produce significant decoherence. We study spin decoherence decay as a function of donor concentration, 29Si concentration, and temperature using cluster expansion techniques specifically adapted to the problem of a sparse dipolarly coupled electron spin bath. Our results agree with the existing experimental spin echo data in Si:P and establish the importance of background dopants as the ultimate decoherence mechanism in isotope-enriched silicon.