On the impact of swirl on the growth of coherent structures