Can neutron stars constrain dark matter