A PARALLEL ALGORITHM FOR NONNEGATIVE MATRIX FACTORIZATION BASED ON NEWTON ITERATION

Nonnegative Matrix Factorization (NMF) is a technique to approximate a nonnegative matrix as a product of two smaller nonnegative matrices. The guaranteed nonnegativity of the factors allows interpreting the approximation as an additive combination of features, a distinctive property that other widely used matrix factorization methods do not have. Several advanced methods for computing this factorization exist, in this paper, we will focus on an algorithm based on Newton iteration. It was adapted for parallel execution on computer clusters, utilizing the message passing communication paradigm. Results of multiple experiments, which were run on a system with up to 80 processor cores, will be presented. We used images as input matrices, which makes it possible to get a visual impression of the NMF approximation quality. The measurements show that for sufficiently large workloads, the parallelized Newton iteration algorithm achieves an almost linear speedup, which makes it a promising candidate for large-scale NMF computations.

[1]  Arto Kaarna,et al.  Non-negative Matrix Factorization Features from Spectral Signatures of AVIRIS Images , 2006, 2006 IEEE International Symposium on Geoscience and Remote Sensing.

[2]  Christos Boutsidis,et al.  SVD based initialization: A head start for nonnegative matrix factorization , 2008, Pattern Recognit..

[3]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[4]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[5]  Karthik Devarajan,et al.  Nonnegative Matrix Factorization: An Analytical and Interpretive Tool in Computational Biology , 2008, PLoS Comput. Biol..

[6]  Emmanuel Vincent,et al.  Harmonic and inharmonic Nonnegative Matrix Factorization for Polyphonic Pitch transcription , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  V. P. Pauca,et al.  Nonnegative matrix factorization for spectral data analysis , 2006 .

[8]  Jonathan M. Smith,et al.  IDES: An Internet Distance Estimation Service for Large Networks , 2006, IEEE Journal on Selected Areas in Communications.

[9]  Jérôme Idier,et al.  Algorithms for Nonnegative Matrix Factorization with the β-Divergence , 2010, Neural Computation.

[10]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[11]  C. D. Meyer,et al.  Initializations for the Nonnegative Matrix Factorization , 2006 .

[12]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[13]  Yuan Gao,et al.  Improving molecular cancer class discovery through sparse non-negative matrix factorization , 2005 .

[14]  Pablo Tamayo,et al.  Metagenes and molecular pattern discovery using matrix factorization , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[15]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[16]  R. Plemmons,et al.  Optimality, computation, and interpretation of nonnegative matrix factorizations , 2004 .

[17]  Andrzej Cichocki,et al.  Nonnegative Matrix Factorization for Motor Imagery EEG Classification , 2006, ICANN.

[18]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[19]  Andrzej Cichocki,et al.  Csiszár's Divergences for Non-negative Matrix Factorization: Family of New Algorithms , 2006, ICA.

[20]  Tao Li,et al.  The Relationships Among Various Nonnegative Matrix Factorization Methods for Clustering , 2006, Sixth International Conference on Data Mining (ICDM'06).

[21]  Tuomas Virtanen,et al.  Monaural Sound Source Separation by Nonnegative Matrix Factorization With Temporal Continuity and Sparseness Criteria , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[22]  P. Smaragdis,et al.  Non-negative matrix factorization for polyphonic music transcription , 2003, 2003 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (IEEE Cat. No.03TH8684).

[23]  Michael W. Berry,et al.  Document clustering using nonnegative matrix factorization , 2006, Inf. Process. Manag..

[24]  Michael W. Berry,et al.  Text Mining Using Non-Negative Matrix Factorizations , 2004, SDM.

[25]  Michael T. Heath,et al.  Scientific Computing: An Introductory Survey , 1996 .

[26]  Anastasios Tefas,et al.  Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification , 2006, IEEE Transactions on Neural Networks.

[27]  Nicolas Gillis,et al.  Document classification using nonnegative matrix factorization and underapproximation , 2009, 2009 IEEE International Symposium on Circuits and Systems.