Spatiotemporal Clustering with Neyman-Scott Processes via Connections to Bayesian Nonparametric Mixture Models

Neyman-Scott processes (NSPs) are point process models that generate clusters of points in time or space. They are natural models for a wide range of phenomena, ranging from neural spike trains to document streams. The clustering property is achieved via a doubly stochastic formulation: first, a set of latent events is drawn from a Poisson process; then, each latent event generates a set of observed data points according to another Poisson process. This construction is similar to Bayesian nonparametric mixture models like the Dirichlet process mixture model (DPMM) in that the number of latent events (i.e. clusters) is a random variable, but the point process formulation makes the NSP especially well suited to modeling spatiotemporal data. While many specialized algorithms have been developed for DPMMs, comparatively fewer works have focused on inference in NSPs. Here, we present novel connections between NSPs and DPMMs, with the key link being a third class of Bayesian mixture models called mixture of finite mixture models (MFMMs). Leveraging this connection, we adapt the standard collapsed Gibbs sampling algorithm for DPMMs to enable scalable Bayesian inference on NSP models. We demonstrate the potential of Neyman-Scott processes on a variety of applications including sequence detection in neural spike trains and event detection in document streams.

[1]  Eric T. Shea-Brown,et al.  The Multivariate Hawkes Process in High Dimensions: Beyond Mutual Excitation , 2017, 1707.04928.

[2]  Matthias Kirchner,et al.  A nonparametric estimation procedure for the Hawkes process: comparison with maximum likelihood estimation , 2018 .

[3]  J. Møller,et al.  Log Gaussian Cox Processes , 1998 .

[4]  R. Wolpert,et al.  Poisson/gamma random field models for spatial statistics , 1998 .

[5]  Michael S. Rosenberg,et al.  Handbook of spatial point-pattern analysis in ecology , 2015, Int. J. Geogr. Inf. Sci..

[6]  P. Green,et al.  Trans-dimensional Markov chain Monte Carlo , 2000 .

[7]  Alan Edelman,et al.  Julia: A Fresh Approach to Numerical Computing , 2014, SIAM Rev..

[8]  Scott W. Linderman,et al.  Poisson-Randomized Gamma Dynamical Systems , 2019, NeurIPS.

[9]  A. Fearnside Bayesian analysis of finite mixture distributions using the allocation sampler , 2007 .

[10]  R. Waagepetersen An Estimating Function Approach to Inference for Inhomogeneous Neyman–Scott Processes , 2007, Biometrics.

[11]  J. Pitman,et al.  Exchangeable Gibbs partitions and Stirling triangles , 2004, math/0412494.

[12]  Luca Vogt Statistics For Spatial Data , 2016 .

[13]  Paris Smaragdis,et al.  Convolutive Speech Bases and Their Application to Supervised Speech Separation , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[14]  D. Blackwell,et al.  Ferguson Distributions Via Polya Urn Schemes , 1973 .

[15]  Peter F. Halpin,et al.  AN EM ALGORITHM FOR HAWKES PROCESS , 2012 .

[16]  Jeffrey W. Miller,et al.  Mixture Models With a Prior on the Number of Components , 2015, Journal of the American Statistical Association.

[17]  J. Møller,et al.  Statistical Inference and Simulation for Spatial Point Processes , 2003 .

[18]  C. Antoniak Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[19]  Scott W. Linderman,et al.  Point process models for sequence detection in high-dimensional neural spike trains , 2020, NeurIPS.

[20]  Mark S. Goldman,et al.  Memory without Feedback in a Neural Network , 2009, Neuron.

[21]  Yosihiko Ogata,et al.  Identification and estimation of superposed Neyman–Scott spatial cluster processes , 2014 .

[22]  Kamran Diba,et al.  Uncovering temporal structure in hippocampal output patterns , 2018, bioRxiv.

[23]  George E. Tita,et al.  Self-Exciting Point Process Modeling of Crime , 2011 .

[24]  J. Davis Univariate Discrete Distributions , 2006 .

[25]  Dietrich Stoyan,et al.  Estimating Pair Correlation Functions of Planar Cluster Processes , 1996 .

[26]  Galin L. Jones,et al.  Markov Chain Monte Carlo in Practice. , 2021, Annual review of public health.

[27]  Sonja Grün,et al.  Methods for identification of spike patterns in massively parallel spike trains , 2018, Biological Cybernetics.

[28]  Le Song,et al.  Learning Social Infectivity in Sparse Low-rank Networks Using Multi-dimensional Hawkes Processes , 2013, AISTATS.

[29]  Katherine A. Heller,et al.  Modelling Reciprocating Relationships with Hawkes Processes , 2012, NIPS.

[30]  M. Thomas A generalization of Poisson's binomial limit for use in ecology. , 1949, Biometrika.

[31]  David M. Blei,et al.  Detecting and Characterizing Events , 2016, EMNLP.

[32]  J. Pitman,et al.  The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator , 1997 .

[33]  Dietrich Stoyan,et al.  Parameter Estimation and Model Selection for Neyman‐Scott Point Processes , 2008, Biometrical journal. Biometrische Zeitschrift.

[34]  Surya Ganguli,et al.  Discovering Precise Temporal Patterns in Large-Scale Neural Recordings through Robust and Interpretable Time Warping , 2019, Neuron.

[35]  X. Liu,et al.  Hawkes process modeling of COVID-19 with mobility leading indicators and spatial covariates , 2020, International Journal of Forecasting.

[36]  Radford M. Neal,et al.  A Split-Merge Markov chain Monte Carlo Procedure for the Dirichlet Process Mixture Model , 2004 .

[37]  Scott W. Linderman,et al.  Discovering Latent Network Structure in Point Process Data , 2014, ICML.

[38]  Renaud Lambiotte,et al.  TiDeH: Time-Dependent Hawkes Process for Predicting Retweet Dynamics , 2016, ICWSM.

[39]  A self-exciting point process to study multicellular spatial signaling patterns , 2021, Proceedings of the National Academy of Sciences.

[40]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .

[41]  Michael,et al.  On a Class of Bayesian Nonparametric Estimates : I . Density Estimates , 2008 .

[42]  C. Shelton,et al.  Deep Neyman-Scott Processes , 2021, AISTATS.

[43]  Albert Y. Lo,et al.  On a Class of Bayesian Nonparametric Estimates: I. Density Estimates , 1984 .

[44]  Daniel Durstewitz,et al.  Sparse convolutional coding for neuronal assembly detection , 2017, NIPS.

[45]  Matthew A. Wilson,et al.  Hippocampal Replay of Extended Experience , 2009, Neuron.

[46]  D. Stoyan,et al.  Recent applications of point process methods in forestry statistics , 2000 .

[47]  Jason Eisner,et al.  The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process , 2016, NIPS.

[48]  J. Neyman,et al.  Statistical Approach to Problems of Cosmology , 1958 .

[49]  K. D. Punta,et al.  An ultra-sparse code underlies the generation of neural sequences in a songbird , 2002 .

[50]  Daniel Durstewitz,et al.  Cell assemblies at multiple time scales with arbitrary lag constellations , 2017, eLife.

[51]  D. Cox Some Statistical Methods Connected with Series of Events , 1955 .

[52]  M. Bartlett The Spectral Analysis of Point Processes , 1963 .

[53]  Rebecca Willett,et al.  Hawkes Process Modeling of Adverse Drug Reactions with Longitudinal Observational Data , 2017, MLHC.

[54]  Jing Wu,et al.  Markov-Modulated Hawkes Processes for Sporadic and Bursty Event Occurrences , 2019 .

[55]  Jeffrey D. Scargle,et al.  An Introduction to the Theory of Point Processes, Vol. I: Elementary Theory and Methods , 2004, Technometrics.

[56]  Rui Zhang,et al.  Variational Inference for Sparse Gaussian Process Modulated Hawkes Process , 2019, AAAI.

[57]  Surya Ganguli,et al.  Fast Convolutive Nonnegative Matrix Factorization Through Coordinate and Block Coordinate Updates , 2019, ArXiv.

[58]  J. Rasmussen Bayesian Inference for Hawkes Processes , 2013 .

[59]  Radford M. Neal Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[60]  Alex H. Williams,et al.  Unsupervised discovery of temporal sequences in high-dimensional datasets, with applications to neuroscience , 2018, bioRxiv.

[61]  Emine Yilmaz,et al.  Self-Attentive Hawkes Process , 2020, ICML.

[62]  Michael I. Jordan,et al.  Modeling Events with Cascades of Poisson Processes , 2010, UAI.

[63]  Trisalyn A. Nelson,et al.  Statistical analysis of spatial and spatio-temporal point patterns, Third Edition, by Peter J. Diggle, Boca Raton, FL, CRC Press, 2013, 263 pp., $49.99, $79.95 EUR 62, 38 (hardback), ISBN 13:978-1-4665-6023-9 , 2015, Int. J. Geogr. Inf. Sci..

[64]  S. Wold Cross-Validatory Estimation of the Number of Components in Factor and Principal Components Models , 1978 .

[65]  M. Howell,et al.  From Reliable Sources: An Introduction to Historical Methods , 2001 .

[66]  Rajeev R. Raje,et al.  Improving social harm indices with a modulated Hawkes process , 2018 .

[67]  Tanmoy Chakraborty,et al.  HawkesEye: Detecting Fake Retweeters Using Hawkes Process and Topic Modeling , 2020, IEEE Transactions on Information Forensics and Security.

[68]  P. Müller,et al.  Chapter 5: Dependent Dirichlet Processes and Other Extensions , 2013 .

[69]  B. Ripley Modelling Spatial Patterns , 1977 .

[70]  M. Abeles,et al.  Detecting precise firing sequences in experimental data , 2001, Journal of Neuroscience Methods.

[71]  G. Torrisi,et al.  A time-modulated Hawkes process to model the spread of COVID-19 and the impact of countermeasures , 2021, Annual Reviews in Control.

[72]  A. Hawkes Spectra of some self-exciting and mutually exciting point processes , 1971 .

[73]  Virgilio Gómez-Rubio,et al.  Spatial Point Patterns: Methodology and Applications with R , 2016 .

[74]  Hongyuan Zha,et al.  Transformer Hawkes Process , 2020, ICML.

[75]  Ting Wang,et al.  Markov-modulated Hawkes process with stepwise decay , 2012 .

[76]  Radford M. Neal,et al.  Splitting and merging components of a nonconjugate Dirichlet process mixture model , 2007 .

[77]  Radford M. Neal Bayesian Mixture Modeling , 1992 .

[78]  A. Brix Generalized Gamma measures and shot-noise Cox processes , 1999, Advances in Applied Probability.

[79]  Walter R. Gilks,et al.  Bayesian model comparison via jump diffusions , 1995 .

[80]  Shuang Li,et al.  COEVOLVE: A Joint Point Process Model for Information Diffusion and Network Co-evolution , 2015, NIPS.

[81]  G. Buzsáki,et al.  Space and Time: The Hippocampus as a Sequence Generator , 2018, Trends in Cognitive Sciences.

[82]  Katherine A. Heller,et al.  The Bayesian Echo Chamber: Modeling Social Influence via Linguistic Accommodation , 2015, AISTATS.

[83]  Zhen Qin,et al.  Hawkes Process Inference With Missing Data , 2018, AAAI.

[84]  Ali Shojaie,et al.  Nearly assumptionless screening for the mutually-exciting multivariate Hawkes process. , 2017, Electronic journal of statistics.

[85]  Yanjie Fu,et al.  EduHawkes: A Neural Hawkes Process Approach for Online Study Behavior Modeling , 2021, SDM.

[86]  Didier Sornette,et al.  The Hawkes process with renewal immigration & its estimation with an EM algorithm , 2016, Comput. Stat. Data Anal..

[87]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[88]  Le Song,et al.  Constructing Disease Network and Temporal Progression Model via Context-Sensitive Hawkes Process , 2015, 2015 IEEE International Conference on Data Mining.

[89]  Ryan P. Adams,et al.  Tractable nonparametric Bayesian inference in Poisson processes with Gaussian process intensities , 2009, ICML '09.

[90]  W. J. Hall,et al.  ON CHARACTERIZATION OF THE GAMMA DISTRIBUTION. , 1968 .

[91]  Guihai Chen,et al.  FM-Hawkes: A Hawkes Process Based Approach for Modeling Online Activity Correlations , 2017, CIKM.

[92]  Michael I. Jordan,et al.  Variational inference for Dirichlet process mixtures , 2006 .

[93]  James R. Foulds,et al.  HawkesTopic: A Joint Model for Network Inference and Topic Modeling from Text-Based Cascades , 2015, ICML.

[94]  Jure Leskovec,et al.  Inferring networks of diffusion and influence , 2010, KDD.

[95]  Ryan P. Adams,et al.  Patterns of Scalable Bayesian Inference , 2016, Found. Trends Mach. Learn..

[96]  S. MacEachern Estimating normal means with a conjugate style dirichlet process prior , 1994 .

[97]  Ole Winther,et al.  Multivariate Hawkes process models of the occurrence of regulatory elements , 2010, BMC Bioinformatics.

[98]  Håkon Toftaker,et al.  Geometric Anisotropic Spatial Point Pattern Analysis and Cox Processes , 2014 .