Toward a model-independent constraint of the high-density dependence of the symmetry energy