The cross-correlation between galaxies of different luminosities and colors