The formation of ultra-compact dwarf galaxies and nucleated dwarf galaxies