Nonnegative Matrix Factorization Using Nonnegative Polynomial Approximations

Nonnegative matrix factorization is a key tool in many data analysis applications such as feature extraction, compression, and noise filtering. Many existing algorithms impose additional constraints to take into account prior knowledge and to improve the physical interpretation. This letter proposes a novel algorithm for nonnegative matrix factorization, in which the factors are modeled by nonnegative polynomials. Using a parametric representation of finite-interval nonnegative polynomials, we obtain an optimization problem without external nonnegativity constraints, which can be solved using conventional quasi-Newton or nonlinear least-squares methods. The polynomial model guarantees smooth solutions and may realize a noise reduction. A dedicated orthogonal compression enables a significant reduction of the matrix dimensions, without sacrificing accuracy. The overall approach scales well to large matrices. The approach is illustrated with applications in hyperspectral imaging and chemical shift brain imaging.

[1]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[2]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[3]  Lieven De Lathauwer,et al.  Stochastic and Deterministic Tensorization for Blind Signal Separation , 2015, LVA/ICA.

[4]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[5]  Pierre Comon,et al.  Handbook of Blind Source Separation: Independent Component Analysis and Applications , 2010 .

[6]  Stephen A. Vavasis,et al.  On the Complexity of Nonnegative Matrix Factorization , 2007, SIAM J. Optim..

[7]  M. V. Van Benthem,et al.  Fast algorithm for the solution of large‐scale non‐negativity‐constrained least squares problems , 2004 .

[8]  B. Reznick,et al.  Polynomials that are positive on an interval , 2000 .

[9]  Lieven De Lathauwer,et al.  Unconstrained Optimization of Real Functions in Complex Variables , 2012, SIAM J. Optim..

[10]  Victoria Stodden,et al.  When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts? , 2003, NIPS.

[11]  Lu Li,et al.  Smooth incomplete matrix factorization and its applications in image/video denoising , 2013, Neurocomputing.

[12]  Lieven De Lathauwer,et al.  Structured Data Fusion , 2015, IEEE Journal of Selected Topics in Signal Processing.

[13]  Lieven De Lathauwer,et al.  Löwner-Based Blind Signal Separation of Rational Functions With Applications , 2016, IEEE Transactions on Signal Processing.

[14]  Hyunsoo Kim,et al.  Sparse Non-negative Matrix Factorizations via Alternating Non-negativity-constrained Least Squares , 2006 .

[15]  Tamara G. Kolda,et al.  Scalable Tensor Factorizations for Incomplete Data , 2010, ArXiv.

[16]  Nicolas Gillis,et al.  Accelerated Multiplicative Updates and Hierarchical ALS Algorithms for Nonnegative Matrix Factorization , 2011, Neural Computation.

[17]  Antonio J. Plaza,et al.  A quantitative and comparative analysis of endmember extraction algorithms from hyperspectral data , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Lucas C. Parra,et al.  Nonnegative matrix factorization for rapid recovery of constituent spectra in magnetic resonance chemical shift imaging of the brain , 2004, IEEE Transactions on Medical Imaging.

[19]  Hyunsoo Kim,et al.  Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method , 2008, SIAM J. Matrix Anal. Appl..

[20]  Nicolas Gillis.  Nonnegative matrix factorization : complexity, algorithms and applications , 2011 .

[21]  Andrzej Cichocki,et al.  Constrained non-Negative Matrix Factorization Method for EEG Analysis in Early Detection of Alzheimer Disease , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[22]  Yukihiko Yamashita,et al.  Smooth nonnegative matrix and tensor factorizations for robust multi-way data analysis , 2015, Signal Process..

[23]  Stefan M. Wild,et al.  Improving non-negative matrix factorizations through structured initialization , 2004, Pattern Recognit..

[24]  Christos Boutsidis,et al.  SVD based initialization: A head start for nonnegative matrix factorization , 2008, Pattern Recognit..

[25]  Lieven De Lathauwer,et al.  A Tensor-Based Method for Large-Scale Blind Source Separation Using Segmentation , 2017, IEEE Transactions on Signal Processing.

[26]  Mikkel N. Schmidt,et al.  Nonnegative Matrix Factorization with Gaussian Process Priors , 2008, Comput. Intell. Neurosci..

[27]  Ioannis Pitas,et al.  Nonnegative Matrix Factorization in Polynomial Feature Space , 2008, IEEE Transactions on Neural Networks.

[28]  R. Plemmons,et al.  Optimality, computation, and interpretation of nonnegative matrix factorizations , 2004 .

[29]  Andrzej Cichocki,et al.  Hierarchical ALS Algorithms for Nonnegative Matrix and 3D Tensor Factorization , 2007, ICA.

[30]  Lieven De Lathauwer,et al.  Blind Separation of Exponential Polynomials and the Decomposition of a Tensor in Rank-(Lr, Lr, 1) Terms , 2011, SIAM J. Matrix Anal. Appl..

[31]  J. Eggert,et al.  Sparse coding and NMF , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[32]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[33]  Seungjin Choi,et al.  Algorithms for orthogonal nonnegative matrix factorization , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[34]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[35]  Nicolas Gillis,et al.  The Why and How of Nonnegative Matrix Factorization , 2014, ArXiv.

[36]  C. D. Meyer,et al.  Initializations for the Nonnegative Matrix Factorization , 2006 .

[37]  M. Fekete.  Proof of three propositions of Paley , 1935 .

[38]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[39]  T. Brown,et al.  A new method for spectral decomposition using a bilinear Bayesian approach. , 1999, Journal of magnetic resonance.

[40]  Chein-I Chang,et al.  An information-theoretic approach to spectral variability, similarity, and discrimination for hyperspectral image analysis , 2000, IEEE Trans. Inf. Theory.

[41]  Nicolas Gillis,et al.  Sparse and unique nonnegative matrix factorization through data preprocessing , 2012, J. Mach. Learn. Res..