THE WEAK LENSING SIGNAL AND THE CLUSTERING OF BOSS GALAXIES. II. ASTROPHYSICAL AND COSMOLOGICAL CONSTRAINTS