Instability of helical tip vortices in rotor wakes