The luminosity function, halo masses and stellar masses of luminous Lyman-break galaxies at redshifts 5 < z < 6