Improvements in Approximation Performance and Parallelization of Nonnegative Matrix Factorization with Newton Iteration

The goal of Nonnegative Matrix Factorization (NMF) is to represent a large nonnegative matrix in an approximate way as a product of two significantly smaller nonnega- tive matrices. In comparison to other algorithms to calculate the NMF, Newton-type methods can be parallelized very well because Newton iterations can be performed in parallel without exchanging data between processes. However, these methods can show problematic convergence behavior, limiting their efficiency. We present a modified algorithm that achieves stable convergence by using Karush-Kuhn-Tucker (KKT) conditions and a reflective technique for constraint handling, backtracking line search for global convergence, and a modified target function to avoid explicit inequality handling. Our method allows for an inexact approach, where only few Newton iterations are performed per outer iteration. Experiments show that this leads to faster convergence in the sequential as well as in the parallel case. Although shorter outer iterations increase communication overhead, speedups are still satisfactory.