The goal of Nonnegative Matrix Factorization (NMF) is to represent a large nonnegative matrix in an approximate way as a product of two significantly smaller nonnega- tive matrices. In comparison to other algorithms to calculate the NMF, Newton-type methods can be parallelized very well because Newton iterations can be performed in parallel without exchanging data between processes. However, these methods can show problematic convergence behavior, limiting their efficiency. We present a modified algorithm that achieves stable convergence by using Karush-Kuhn-Tucker (KKT) conditions and a reflective technique for constraint handling, backtracking line search for global convergence, and a modified target function to avoid explicit inequality handling. Our method allows for an inexact approach, where only few Newton iterations are performed per outer iteration. Experiments show that this leads to faster convergence in the sequential as well as in the parallel case. Although shorter outer iterations increase communication overhead, speedups are still satisfactory.
[1]
Inderjit S. Dhillon,et al.
Fast Newton-type Methods for the Least Squares Nonnegative Matrix Approximation Problem
,
2007,
SDM.
[2]
Thomas F. Coleman,et al.
An Interior Trust Region Approach for Nonlinear Minimization Subject to Bounds
,
1993,
SIAM J. Optim..
[3]
H. Sebastian Seung,et al.
Learning the parts of objects by non-negative matrix factorization
,
1999,
Nature.
[4]
Marián Vajtersic,et al.
Parallel Nonnegative Matrix Factorization via Newton Iteration
,
2016,
Parallel Process. Lett..
[5]
Stefania Bellavia,et al.
An interior point Newton‐like method for non‐negative least‐squares problems with degenerate solution
,
2006,
Numer. Linear Algebra Appl..