The Growth of Galaxies in Cosmological Simulations of Structure Formation