Bifurcation of coupled-mode responses by modal coupling in cubic nonlinear systems