Steering interchange of polariton branches via coherent and incoherent dynamics