Theory of mode interaction in the gyrotron