The influence of halo assembly on galaxies and galaxy groups