On nonlinear modes of systems with internal resonance