Parallel Nonnegative Matrix Factorization Algorithm on the Distributed Memory Platform

Nonnegative matrix factorization provides a new sight into the observed signals and has been extensively applied in face recognition, text mining and spectral data analysis. Despite the success, it is inefficient for the large-scale data set, due to the notoriously slow convergence of the multiplicative updating method. In this paper, we try to solve the problem through the parallel computing technique. Considering the limitation of the shared memory platform, the parallel algorithms are implemented on the distributed memory platform with the message passing interface library. Moreover, we adopt the two-layer cascade factorization strategy to eliminate the network consumption. The parallel implementations are evaluated on a 16-node Beowulf cluster with two data sets in different scale. The experiments demonstrate that the proposed method is effective in both precision and efficiency.

[1]  Andrzej Cichocki,et al.  Non-negative Matrix Factorization with Quasi-Newton Optimization , 2006, ICAISC.

[2]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[3]  Message P Forum,et al.  MPI: A Message-Passing Interface Standard , 1994 .

[4]  Robert O. Green,et al.  Airborne visible/infrared imaging spectrometer (AVIRIS): recent improvements to the sensor and data facility , 1993, Defense, Security, and Sensing.

[5]  José M. Bioucas-Dias,et al.  Vertex component analysis: a fast algorithm to unmix hyperspectral data , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Rajeev Thakur,et al.  Optimization of Collective Communication Operations in MPICH , 2005, Int. J. High Perform. Comput. Appl..

[7]  Andrzej Cichocki,et al.  Multilayer Nonnegative Matrix Factorization Using Projected Gradient Approaches , 2007, Int. J. Neural Syst..

[8]  Christos Boutsidis,et al.  SVD based initialization: A head start for nonnegative matrix factorization , 2008, Pattern Recognit..

[9]  Alexander G. Gray,et al.  A I ] 2 2 A pr 2 00 9 Non-Negative Matrix Factorization , Convexity and Isometry ∗ Nikolaos Vasiloglou , 2009 .

[10]  Michael W. Berry,et al.  Email Surveillance Using Non-negative Matrix Factorization , 2005, Comput. Math. Organ. Theory.

[11]  R. Clark,et al.  The U. S. Geological Survey, Digital Spectral Library: Version 1 (0.2 to 3.0um) , 1993 .

[12]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[13]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[14]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[15]  Roger N. Clark,et al.  The US Geological Survey, digital spectral reflectance library: version 1: 0.2 to 3.0 microns , 1993 .

[16]  Fred A. Kruse,et al.  Evaluation and validation of EO-1 Hyperion for geologic mapping , 2002, IEEE International Geoscience and Remote Sensing Symposium.

[17]  Stefan M. Wild.  Seeding Non-Negative Matrix Factorizations with the Spherical K-Means Clustering , 2003 .

[18]  Lukasz G. Maciak,et al.  Sequential and Parallel Feature Extraction in Hyperspectral Data Using Nonnegative Matrix Factorization , 2007, 2007 IEEE Long Island Systems, Applications and Technology Conference.

[19]  Hao Zhou,et al.  Blind decomposition of mixed pixels using constrained non-negative matrix factorization , 2005, Proceedings. 2005 IEEE International Geoscience and Remote Sensing Symposium, 2005. IGARSS '05..

[20]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[21]  Mario Winter,et al.  N-FINDR: an algorithm for fast autonomous spectral end-member determination in hyperspectral data , 1999, Optics & Photonics.

[22]  A. Cichocki,et al.  Multilayer nonnegative matrix factorisation , 2006 .

[23]  Feng Qianjin,et al.  Projected gradient methods for Non-negative Matrix Factorization based relevance feedback algorithm in medical image retrieval , 2011 .

[24]  Stefan M. Wild,et al.  Improving non-negative matrix factorizations through structured initialization , 2004, Pattern Recognit..

[25]  V. P. Pauca,et al.  Nonnegative matrix factorization for spectral data analysis , 2006 .

[26]  Arto Kaarna,et al.  Non-negative Matrix Factorization Features from Spectral Signatures of AVIRIS Images , 2006, 2006 IEEE International Symposium on Geoscience and Remote Sensing.

[27]  Hairong Qi,et al.  A Constrained Non-Negative Matrix Factorization Approach to Unmix Highly Mixed Hyperspectral Data , 2007, 2007 IEEE International Conference on Image Processing.

[28]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[29]  Stefan A. Robila,et al.  A parallel unmixing algorithm for hyperspectral images , 2006, SPIE Optics East.

[30]  Yin Zhang,et al.  Accelerating the Lee-Seung Algorithm for Nonnegative Matrix Factorization , 2005 .