Fundamental limits and near-optimal design of graphene modulators and non-reciprocal devices