We study the prospects for detection of continuous gravitational signals from"normal"Galactic neutron stars, i.e. non-recycled ones. We use a synthetic population generated by evolving stellar remnants in time, according to several models. We consider the most recent constraints set by all-sky searches for continuous gravitational waves and use them for our detectability criteria. We discuss detection prospects for the current and the next generation of gravitational wave detectors. We find that neutron stars whose ellipticity is solely caused by magnetic deformations cannot produce any detectable signal, not even by 3rd-generation detectors. Currently detectable sources all have $B\lesssim10^{12}$ G and deformations not solely due to the magnetic field. For these in fact we find that the larger the magnetic field is, the larger is the ellipticity required for the signal to be detectable and this ellipticity is well above the value induced by the magnetic field. Third-generation detectors as the Einstein Telescope and Cosmic Explorer will be able to detect up to $\approx 250$ more sources than current detectors. We briefly treat the case of recycled neutron stars, with a simplified model. We find that continuous gravitational waves from these objects will likely remain elusive to detection by current detectors but should be detectable with the next generation of detectors.