Absence of confinement in the absence of vortices