Defect energies of two-, three- and four-dimensional Ising spin glasses