Matrix iteration method for nonlinear eigenvalue problems with applications