$^1$S$_0$ Neutron Pairing vs Observations of Cooling Neutron Stars