The CORALIE survey for southern extrasolar planets. XIX. Brown dwarfs and stellar companions unveiled by radial velocity and astrometry