Streamwise vortices in shear flows: harbingers of transition and the skeleton of coherent structures