Fast Convolutive Nonnegative Matrix Factorization Through Coordinate and Block Coordinate Updates

Identifying recurring patterns in high-dimensional time series data is an important problem in many scientific domains. A popular model to achieve this is convolutive nonnegative matrix factorization (CNMF), which extends classic nonnegative matrix factorization (NMF) to extract short-lived temporal motifs from a long time series. Prior work has typically fit this model by multiplicative parameter updates---an approach widely considered to be suboptimal for NMF, especially in large-scale data applications. Here, we describe how to extend two popular and computationally scalable NMF algorithms---Hierarchical Alternating Least Squares (HALS) and Alternatining Nonnegative Least Squares (ANLS)---for the CNMF model. Both methods demonstrate performance advantages over multiplicative updates on large-scale synthetic and real world data.

[1]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[2]  Haesun Park,et al.  Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework , 2014, J. Glob. Optim..

[3]  P. Smaragdis,et al.  Non-negative matrix factorization for polyphonic music transcription , 2003, 2003 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (IEEE Cat. No.03TH8684).

[4]  Paris Smaragdis,et al.  Convolutive Speech Bases and Their Application to Supervised Speech Separation , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[5]  J. Nathan Kutz,et al.  Randomized nonnegative matrix factorization , 2018, Pattern Recognit. Lett..

[6]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[7]  Stefano Panzeri,et al.  Inference of neuronal functional circuitry with spike-triggered non-negative matrix factorization , 2017, Nature Communications.

[8]  Marinka Zitnik,et al.  NIMFA: A Python Library for Nonnegative Matrix Factorization , 2012, J. Mach. Learn. Res..

[9]  Haesun Park,et al.  Fast Nonnegative Matrix Factorization: An Active-Set-Like Method and Comparisons , 2011, SIAM J. Sci. Comput..

[10]  Sanjeev Arora,et al.  Computing a nonnegative matrix factorization -- provably , 2011, STOC '12.

[11]  Stephen J. Wright.  Coordinate descent algorithms , 2015, Mathematical Programming.

[12]  G. Buzsáki,et al.  Behavior-dependent short-term assembly dynamics in the medial prefrontal cortex , 2008, Nature Neuroscience.

[13]  Alan Edelman,et al.  Julia: A Fresh Approach to Numerical Computing , 2014, SIAM Rev..

[14]  Nicolas Gillis,et al.  The Why and How of Nonnegative Matrix Factorization , 2014, ArXiv.

[15]  Richard Hans Robert Hahnloser,et al.  An ultra-sparse code underliesthe generation of neural sequences in a songbird , 2002, Nature.

[16]  Andrzej Cichocki,et al.  Hierarchical ALS Algorithms for Nonnegative Matrix and 3D Tensor Factorization , 2007, ICA.

[17]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[18]  Amir Beck,et al.  On the Convergence of Block Coordinate Descent Type Methods , 2013, SIAM J. Optim..

[19]  Paris Smaragdis,et al.  Non-negative Matrix Factor Deconvolution; Extraction of Multiple Sound Sources from Monophonic Inputs , 2004, ICA.

[20]  Barak A. Pearlmutter,et al.  Discovering Convolutive Speech Phones Using Sparseness and Non-negativity , 2007, ICA.

[21]  Sen Jia,et al.  Constrained Nonnegative Matrix Factorization for Hyperspectral Unmixing , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Haesun Park,et al.  A high-performance parallel algorithm for nonnegative matrix factorization , 2015, PPoPP.

[23]  Rafael Yuste,et al.  Fast nonnegative deconvolution for spike train inference from population calcium imaging. , 2009, Journal of neurophysiology.

[24]  Alex H. Williams,et al.  Unsupervised discovery of temporal sequences in high-dimensional datasets, with applications to neuroscience , 2018, bioRxiv.

[25]  Athanasios Katsamanis,et al.  Automatic Data-Driven Learning of Articulatory Primitives from Real-Time MRI Data Using Convolutive NMF with Sparseness Constraints , 2011, INTERSPEECH.

[26]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[27]  Liang Tao,et al.  Unsupervised learning of phonemes of whispered speech in a noisy environment based on convolutive non-negative matrix factorization , 2014, Inf. Sci..

[28]  Victoria Stodden,et al.  When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts? , 2003, NIPS.

[29]  Stephen A. Vavasis,et al.  On the Complexity of Nonnegative Matrix Factorization , 2007, SIAM J. Optim..