How to Apply Random Projections to Nonnegative Matrix Factorization with Missing Entries?

Random projections belong to the major techniques to process big data and have been successfully applied to Nonnegative Matrix Factorization (NMF). However, they cannot be applied in the case of missing entries in the matrix to factorize, which occurs in many actual problems with large data matrices. In this paper, we thus aim to solve this issue and we propose a novel framework to apply random projections in weighted NMF, where the weight models the confidence in the data (or the absence of confidence in the case of missing data). We experimentally show the proposed framework to significantly speed-up state-of-the-art NMF methods under some mild conditions. In particular, the proposed strategy is particularly efficient when combined with Nesterov gradient or alternating least squares.

[1]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[2]  Matthieu Puigt,et al.  Fast Nonnegative Matrix Factorization and Completion Using Nesterov Iterations , 2017, LVA/ICA.

[3]  Chao Liu,et al.  Distributed nonnegative matrix factorization for web-scale dyadic data analysis on mapreduce , 2010, WWW '10.

[4]  Guillermo Sapiro,et al.  Compressed Nonnegative Matrix Factorization Is Fast and Accurate , 2015, IEEE Transactions on Signal Processing.

[5]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..

[6]  Farouk Yahaya,et al.  Faster-than-fast NMF using random projections and Nesterov iterations , 2018, ArXiv.

[7]  Fillia Makedon,et al.  Learning from Incomplete Ratings Using Non-negative Matrix Factorization , 2006, SDM.

[8]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[9]  Dan Hu,et al.  Non‐negative Matrix Factorization Algorithms and Applications , 2014 .

[10]  T. Virtanen.  Monaural Sound Source Separation by Perceptually Weighted Non-Negative Matrix Factorization , 2003 .

[11]  Florent Krzakala,et al.  Random projections through multiple optical scattering: Approximating Kernels at the speed of light , 2015, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[13]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[14]  Fei Wang,et al.  Efficient Nonnegative Matrix Factorization with Random Projections , 2010, SDM.

[15]  M. Puigt,et al.  Non-negative Matrix Factorization under equality constraints—a study of industrial source identification , 2014 .

[16]  J. Nathan Kutz,et al.  Randomized nonnegative matrix factorization , 2017, Pattern Recognit. Lett..

[17]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[18]  Matthieu Puigt,et al.  Environmental signal processing: new trends and applications , 2017, ESANN.

[19]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[20]  Matthieu Puigt,et al.  Nonlinear mobile sensor calibration using informed semi-nonnegative matrix factorization with a Vandermonde factor , 2016, 2016 IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM).

[21]  Seungjin Choi,et al.  Weighted nonnegative matrix factorization , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[22]  Matthieu Puigt,et al.  Informed Nonnegative Matrix Factorization Methods for Mobile Sensor Network Calibration , 2018, IEEE Transactions on Signal and Information Processing over Networks.

[23]  Yu-Jin Zhang,et al.  Nonnegative Matrix Factorization: A Comprehensive Review , 2013, IEEE Transactions on Knowledge and Data Engineering.

[24]  Zhigang Luo,et al.  NeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization , 2012, IEEE Transactions on Signal Processing.

[25]  Tommi S. Jaakkola,et al.  Weighted Low-Rank Approximations , 2003, ICML.

[26]  Emmanuel Vincent,et al.  Single-channel audio source separation with NMF: divergences, constraints and algorithms , 2018 .

[27]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Alexandre Bernardino,et al.  Unifying Nuclear Norm and Bilinear Factorization Approaches for Low-Rank Matrix Decomposition , 2013, 2013 IEEE International Conference on Computer Vision.

[29]  Antonio J. Plaza,et al.  Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.