Nonlinear dynamics of polarization modulation instability in optical fiber