Iterative improvement of the Multiplicative Update NMF algorithm using nature-inspired optimization

Low-rank approximations of data (e. g. based on the Singular Value Decomposition) have proven very useful in various data mining applications. The Non-negative Matrix Factorization (NMF) leads to special low-rank approximations which satisfy non-negativity constraints. The Multiplicative Update (MU) algorithm is one of the two original NMF algorithms and is still one of the fastest NMF algorithms per iteration. Nevertheless, MU demands a quite large number of iterations in order to provide an accurate approximation of the original data. In this paper we present a new iterative update strategy for the MU algorithm based on nature-inspired optimization algorithms. The goal is to achieve a better accuracy per runtime compared to the standard version of MU. Several properties of the NMF objective function underlying the MU algorithm motivate the utilization of heuristic search algorithms. Indeed, this function is usually non-differentiable, discontinuous, and may possess many local minima. Experimental results show that our new iterative update strategy for the MU algorithm achieves the same approximation error than the standard version in significantly fewer iterations and in faster overall runtime.

[1]  Mikkel N. Schmidt,et al.  Nonnegative Matrix Factorization with Gaussian Process Priors , 2008, Comput. Intell. Neurosci..

[2]  Václav Snásel,et al.  Developing Genetic Algorithms for Boolean Matrix Factorization , 2008, DATESO.

[3]  Wilfried N. Gansterer,et al.  libNMF - A Library for Nonnegative Matrix Factorization , 2011, Comput. Informatics.

[4]  Ying Tan,et al.  Using Population Based Algorithms for Initializing Nonnegative Matrix Factorization , 2011, ICSI.

[5]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[6]  Christos Boutsidis,et al.  SVD based initialization: A head start for nonnegative matrix factorization , 2008, Pattern Recognit..

[7]  Fabian J. Theis,et al.  Sparse Nonnegative Matrix Factorization with Genetic Algorithms for Microarray Analysis , 2007, 2007 International Joint Conference on Neural Networks.

[8]  C. D. Meyer,et al.  Initializations for the Nonnegative Matrix Factorization , 2006 .

[9]  Ying Tan,et al.  Fireworks Algorithm for Optimization , 2010, ICSI.

[10]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[11]  Christine M. Anderson-Cook.  Practical Genetic Algorithms (2nd ed.) , 2005 .

[12]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[13]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[14]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[15]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[16]  Wilfried N. Gansterer,et al.  Utilizing Nonnegative Matrix Factorization for Email Classification Problems , 2010 .

[17]  James Kennedy,et al.  Defining a Standard for Particle Swarm Optimization , 2007, 2007 IEEE Swarm Intelligence Symposium.

[18]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[19]  Hyunsoo Kim,et al.  Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method , 2008, SIAM J. Matrix Anal. Appl..

[20]  C. J. A. B. Filho,et al.  On the influence of the swimming operators in the Fish School Search algorithm , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.