Poisson-Gamma dynamical systems

This paper presents a dynamical system based on the Poisson-Gamma construction for sequentially observed multivariate count data. Inherent to the model is a novel Bayesian nonparametric prior that ties and shrinks parameters in a powerful way. We develop theory about the model's infinite limit and its steady-state. The model's inductive bias is demonstrated on a variety of real-world datasets where it is shown to learn interpretable structure and have superior predictive performance.

[1]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[2]  D. Dunson,et al.  Bayesian latent variable models for mixed discrete outcomes. , 2005, Biostatistics.

[3]  Yoshua Bengio,et al.  Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription , 2012, ICML.

[4]  Roy M. Howard,et al.  Linear System Theory , 1992 .

[5]  S. Haykin.  Kalman Filtering and Neural Networks , 2001 .

[6]  R. Adelson.  Compound Poisson Distributions , 1966 .

[7]  Zoubin Ghahramani,et al.  Learning Nonlinear Dynamical Systems Using an EM Algorithm , 1998, NIPS.

[8]  Jon M. Kleinberg,et al.  Bursty and Hierarchical Structure in Streams , 2002, Data Mining and Knowledge Discovery.

[9]  David M. Blei,et al.  Scalable Recommendation with Poisson Factorization , 2013, ArXiv.

[10]  Gaston H. Gonnet,et al.  On the LambertW function , 1996, Adv. Comput. Math..

[11]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[12]  M. R. Leadbetter.  Poisson Processes , 2011, International Encyclopedia of Statistical Science.

[13]  David B. Dunson,et al.  Beta-Negative Binomial Process and Poisson Factor Analysis , 2011, AISTATS.

[14]  Mingyuan Zhou,et al.  Augmentable Gamma Belief Networks , 2016, J. Mach. Learn. Res..

[15]  Ali Taylan Cemgil,et al.  Bayesian Inference for Nonnegative Matrix Factorisation Models , 2009, Comput. Intell. Neurosci..

[16]  References , 1971 .

[17]  P. McCullagh,et al.  Generalized Linear Models , 1992 .

[18]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[19]  Lawrence Carin,et al.  Negative Binomial Process Count and Mixture Modeling , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  M. Bulmer.  On Fitting the Poisson Lognormal Distribution to Species-Abundance Data , 1974 .

[21]  Sham M. Kakade,et al.  A Linear Dynamical System Model for Text , 2015, ICML.

[22]  Barbara E. Engelhardt,et al.  Hierarchical Compound Poisson Factorization , 2016, ICML.

[23]  Melvin J. Hinich,et al.  Time Series Analysis by State Space Methods , 2001 .

[24]  Joydeep Ghosh,et al.  Nonparametric Bayesian Factor Analysis for Dynamic Count Matrices , 2015, AISTATS.

[25]  Lawrence Carin,et al.  Augment-and-Conquer Negative Binomial Processes , 2012, NIPS.

[26]  Mingyuan Zhou,et al.  Infinite Edge Partition Models for Overlapping Community Detection and Link Prediction , 2015, AISTATS.

[27]  John P. Cunningham,et al.  Empirical models of spiking in neural populations , 2011, NIPS.

[28]  John F. Canny,et al.  GaP: a factor model for discrete data , 2004, SIGIR '04.

[29]  David M. Blei,et al.  Dynamic Poisson Factorization , 2015, RecSys.

[30]  Mingyuan Zhou,et al.  Nonparametric Bayesian Negative Binomial Factor Analysis , 2016, Bayesian Analysis.

[31]  John D. Lafferty,et al.  Dynamic topic models , 2006, ICML.