Social Structure from the Early Census Returns