On the mechanism of instabilities in nonlinear systems