A Comparative Study of Temporal Non-Negative Matrix Factorization with Gamma Markov Chains

Non-negative matrix factorization (NMF) has become a well-established class of methods for the analysis of non-negative data. In particular, a lot of effort has been devoted to probabilistic NMF, namely estimation or inference tasks in probabilistic models describing the data, based for example on Pois-son or exponential likelihoods. When dealing with time series data, several works have proposed to model the evolution of the activation coefficients as a non-negative Markov chain, most of the time in relation with the Gamma distribution, giving rise to so-called temporal NMF models. In this paper, we review three Gamma Markov chains of the NMF literature, and show that they all share the same drawback: the absence of a well-defined stationary distribution. We then introduce a fourth process, an overlooked model of the time series literature named BGAR(1), which overcomes this limitation. These four temporal NMF models are then compared in a MAP framework on a prediction task, in the context of the Poisson likelihood.

[1]  Jonathan Le Roux,et al.  Non-negative dynamical system with application to speech and audio , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[2]  Cédric Févotte,et al.  Majorization-minimization algorithm for smooth Itakura-Saito nonnegative matrix factorization , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  Ali Taylan Cemgil,et al.  Learning the beta-Divergence in Tweedie Compound Poisson Matrix Factorization Models , 2013, ICML.

[4]  Mingyuan Zhou,et al.  Deep Poisson gamma dynamical systems , 2018, NeurIPS.

[5]  Longbing Cao,et al.  Gamma-Poisson Dynamic Matrix Factorization Embedded with Metadata Influence , 2018, NeurIPS.

[6]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[7]  Jérôme Idier,et al.  Algorithms for Nonnegative Matrix Factorization with the β-Divergence , 2010, Neural Computation.

[8]  Mingyuan Zhou,et al.  Poisson-Gamma dynamical systems , 2016, NIPS.

[9]  D. P. Gaver,et al.  First-order autoregressive gamma sequences and point processes , 1980, Advances in Applied Probability.

[10]  Joydeep Ghosh,et al.  A Dual Markov Chain Topic Model for Dynamic Environments , 2018, KDD.

[11]  Joydeep Ghosh,et al.  Nonparametric Bayesian Factor Analysis for Dynamic Count Matrices , 2015, AISTATS.

[12]  Barbara E. Engelhardt,et al.  Hierarchical Compound Poisson Factorization , 2016, ICML.

[13]  Mark Girolami,et al.  Dynamic content based ranking , 2020, AISTATS.

[14]  Nancy Bertin,et al.  Nonnegative Matrix Factorization with the Itakura-Saito Divergence: With Application to Music Analysis , 2009, Neural Computation.

[15]  Simon J. Godsill,et al.  Bayesian extensions to non-negative matrix factorisation for audio signal modelling , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[16]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[17]  John F. Canny,et al.  GaP: a factor model for discrete data , 2004, SIGIR '04.

[18]  Ali Taylan Cemgil,et al.  Conjugate Gamma Markov Random Fields for Modelling Nonstationary Sources , 2007, ICA.

[19]  Ole Winther,et al.  Bayesian Non-negative Matrix Factorization , 2009, ICA.

[20]  David M. Blei,et al.  Scalable Recommendation with Hierarchical Poisson Factorization , 2015, UAI.

[21]  David M. Blei,et al.  Dynamic Poisson Factorization , 2015, RecSys.

[22]  Mingyuan Zhou,et al.  Nonparametric Bayesian Negative Binomial Factor Analysis , 2016, Bayesian Analysis.

[23]  John D. Lafferty,et al.  Dynamic topic models , 2006, ICML.

[24]  Ali Taylan Cemgil,et al.  Bayesian Inference for Nonnegative Matrix Factorisation Models , 2009, Comput. Intell. Neurosci..

[25]  Perry R. Cook,et al.  Bayesian Nonparametric Matrix Factorization for Recorded Music , 2010, ICML.

[26]  ChengYue Gong,et al.  Deep Dynamic Poisson Factorization Model , 2017, NIPS.

[27]  David B. Dunson,et al.  Beta-Negative Binomial Process and Poisson Factor Analysis , 2011, AISTATS.

[28]  D. Hunter,et al.  A Tutorial on MM Algorithms , 2004 .

[29]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[30]  Ghassen Jerfel,et al.  Dynamic Collaborative Filtering With Compound Poisson Factorization , 2016, AISTATS.

[31]  Thomas Oberlin,et al.  Recommendation from Raw Data with Adaptive Compound Poisson Factorization , 2019, UAI.