Do we know the ground vacuum states for nuclear, neutron and alpha matter?