Generalized Pólya Urn for Time-Varying Pitman-Yor Processes

This article introduces a class of first-order stationary time-varying Pitman-Yor processes. Subsuming our construction of time-varying Dirichlet processes presented in (Caron et al., 2007), these models can be used for time-dynamic density estimation and clustering. Our intuitive and simple construction relies on a generalized Polya urn scheme. Significantly, this construction yields marginal distributions at each time point that can be explicitly characterized and easily controlled. Inference is performed using Markov chain Monte Carlo and sequential Monte Carlo methods. We demonstrate our models and algorithms on epidemiological and video tracking data.

[1]  D. Dunson,et al.  Kernel stick-breaking processes. , 2008, Biometrika.

[2]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Sumeetpal S. Singh,et al.  Particle approximations of the score and observed information matrix in state space models with application to parameter estimation , 2011 .

[4]  Anil M. Cheriyadat,et al.  Non-negative matrix factorization of partial track data for motion segmentation , 2010, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Christophe Andrieu,et al.  Particle Markov Chain Monte Carlo for Efficient Numerical Simulation , 2009 .

[6]  Michael I. Jordan,et al.  Bayesian Nonparametrics: Hierarchical Bayesian nonparametric models with applications , 2010 .

[7]  Arnaud Doucet,et al.  An overview of sequential Monte Carlo methods for parameter estimation in general state-space models , 2009 .

[8]  S. Cornell,et al.  Dynamics of the 2001 UK Foot and Mouth Epidemic: Stochastic Dispersal in a Heterogeneous Landscape , 2001, Science.

[9]  J. Lafferty,et al.  Time-Sensitive Dirichlet Process Mixture Models , 2005 .

[10]  Paul Fearnhead,et al.  Particle filters for mixture models with an unknown number of components , 2004, Stat. Comput..

[11]  Alexander Gnedin,et al.  Regenerative Partition Structures , 2005, Electron. J. Comb..

[12]  S. MacEachern,et al.  Bayesian Nonparametric Spatial Modeling With Dirichlet Process Mixing , 2005 .

[13]  S. MacEachern,et al.  An ANOVA Model for Dependent Random Measures , 2004 .

[14]  Hakan Erdogan,et al.  Detecting and Tracking Unknown Number of Objects with Dirichlet Process Mixture Models and Markov Random Fields , 2013, ISVC.

[15]  M. Pitt,et al.  Constructing Stationary Time Series Models Using Auxiliary Variables With Applications , 2005 .

[16]  A. Ellis,et al.  PETS2010 and PETS2009 Evaluation of Results Using Individual Ground Truthed Single Views , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[17]  Stephen G. Walker,et al.  A bivariate Dirichlet process , 2003 .

[18]  Stephen G. Walker,et al.  Constructing First Order Stationary Autoregressive Models via Latent Processes , 2002 .

[19]  N. Pillai,et al.  Bayesian density regression , 2007 .

[20]  Neil J. Gordon,et al.  Editors: Sequential Monte Carlo Methods in Practice , 2001 .

[21]  B Skyrms,et al.  A dynamic model of social network formation. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Peter I. Frazier,et al.  Distance dependent Chinese restaurant processes , 2009, ICML.

[23]  Arnaud Doucet,et al.  Bayesian Inference for Linear Dynamic Models With Dirichlet Process Mixtures , 2007, IEEE Transactions on Signal Processing.

[24]  Ping-Shi Wu,et al.  Time-Dynamic Density and Mode Estimation with Application to Fast Mode Tracking , 2005 .

[25]  M. Li,et al.  Particle Markov chain Monte Carlo methods , 2015 .

[26]  W. Eric L. Grimson,et al.  Trajectory Analysis and Semantic Region Modeling Using Nonparametric Hierarchical Bayesian Models , 2011, International Journal of Computer Vision.

[27]  Satishs Iyengar,et al.  Multivariate Models and Dependence Concepts , 1998 .

[28]  D. Blackwell,et al.  Ferguson Distributions Via Polya Urn Schemes , 1973 .

[29]  Yee Whye Teh,et al.  Modelling Genetic Variations using Fragmentation-Coagulation Processes , 2011, NIPS.

[30]  E. Xing,et al.  Dynamic Non-Parametric Mixture Models and The Recurrent Chinese Restaurant Process a , 2008 .

[31]  Stephen G. Walker,et al.  Slice sampling mixture models , 2011, Stat. Comput..

[32]  Donato Michele Cifarelli,et al.  Nonparametric statistical problems under partial exchangeability . The role of associative means . Translated from Problemi statistici non parametrici in condizioni di scambiabilità parziale : impiego di medie associative , 2008 .

[33]  L. Shapley,et al.  Statistics, probability, and game theory : papers in honor of David Blackwell , 1999 .

[34]  Arnaud Doucet,et al.  Generalized Polya Urn for Time-varying Dirichlet Process Mixtures , 2007, UAI.

[35]  G. Rigoll,et al.  Multi camera person tracking applying a graph-cuts based foreground segmentation in a homography framework , 2009, 2009 Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance.

[36]  David S. Doermann,et al.  Tools and techniques for video performance evaluation , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[37]  Stephen G. Walker,et al.  Sampling the Dirichlet Mixture Model with Slices , 2006, Commun. Stat. Simul. Comput..

[38]  F. Fleuret,et al.  Multiple object tracking using flow linear programming , 2009, 2009 Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance.

[39]  Peter J. Green,et al.  Colouring and breaking sticks: random distributions and heterogeneous clustering , 2010, 1003.3988.

[40]  W. Eric L. Grimson,et al.  Construction of Dependent Dirichlet Processes based on Poisson Processes , 2010, NIPS.

[41]  Michael D. Cahalan,et al.  Bayesian Spatio-Dynamic Modeling in Cell Motility Studies: Learning Nonlinear Taxic Fields Guiding the Immune Response , 2012 .

[42]  Yee Whye Teh,et al.  Dependent Dirichlet Process Spike Sorting , 2008, NIPS.

[43]  Yee Whye Teh,et al.  Spatial Normalized Gamma Processes , 2009, NIPS.

[44]  J. Pitman.  Exchangeable and partially exchangeable random partitions , 1995 .

[45]  Jing Zhang,et al.  Framework for Performance Evaluation of Face, Text, and Vehicle Detection and Tracking in Video: Data, Metrics, and Protocol , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  P. Green,et al.  Modelling Heterogeneity With and Without the Dirichlet Process , 2001 .

[47]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[48]  P. Green,et al.  Probability and Mathematical Genetics: Papers in Honour of Sir John Kingman , 2010 .

[49]  David B Dunson,et al.  Nonparametric Bayesian models through probit stick-breaking processes. , 2011, Bayesian analysis.

[50]  J. Pitman,et al.  The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator , 1997 .

[51]  Purushottam W. Laud,et al.  Bayesian Nonparametric Inference for Random Distributions and Related Functions , 1999 .

[52]  Jun S. Liu,et al.  Sequential importance sampling for nonparametric Bayes models: The next generation , 1999 .

[53]  David Pfau,et al.  Forgetting Counts: Constant Memory Inference for a Dependent Hierarchical Pitman-Yor Process , 2010, ICML.

[54]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[55]  Fernando A. Quintana,et al.  Nonparametric Bayesian data analysis , 2004 .

[56]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[57]  François Septier,et al.  State and impulsive time-varying measurement noise density estimation in nonlinear dynamic systems using Dirichlet Process Mixtures , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[58]  S. Roweis,et al.  Time-Varying Topic Models using Dependent Dirichlet Processes , 2005 .

[59]  Eric P. Xing,et al.  The Dependent Dirichlet Process Mixture of Objects for Detection-free Tracking and Object Modeling , 2014, AISTATS.

[60]  J. E. Griffin,et al.  Order-Based Dependent Dirichlet Processes , 2006 .

[61]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[62]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[63]  S. J. Koopman.  Discussion of `Particle Markov chain Monte Carlo methods – C. Andrieu, A. Doucet and R. Holenstein’ [Review of: Particle Markov chain Monte Carlo methods] , 2010 .

[64]  Xiaoqin Zhang,et al.  Trajectory-Based Video Retrieval Using Dirichlet Process Mixture Models , 2008, BMVC.

[65]  Mübeccel Demirekler,et al.  Dynamic Speech Spectrum Representation and Tracking Variable Number of Vocal Tract Resonance Frequencies With Time-Varying Dirichlet Process Mixture Models , 2009, IEEE Transactions on Audio, Speech, and Language Processing.

[66]  J. Pitman.  Some developments of the Blackwell-MacQueen urn scheme , 1996 .

[67]  Robert T. Collins,et al.  Evaluation of sampling-based pedestrian detection for crowd counting , 2009, 2009 Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance.

[68]  Bruce A. Draper,et al.  Simple real-time human detection using a single correlation filter , 2009, 2009 Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance.

[69]  Luc Van Gool,et al.  Markovian tracking-by-detection from a single, uncalibrated camera , 2009 .

[70]  Frank Wood,et al.  Infinite Structured Hidden Semi-Markov Models , 2014, 1407.0044.

[71]  G. Roberts,et al.  Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models , 2007, 0710.4228.

[72]  Nicholas J. Foti,et al.  A Survey of Non-Exchangeable Priors for Bayesian Nonparametric Models , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[73]  K. Mengersen,et al.  BAYESIAN NONPARAMETRIC DEPENDENT MODEL FOR THE STUDY OF DIVERSITY FOR SPECIES DATA , 2014 .

[74]  J. Sethuraman.  A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[75]  Michael I. Jordan,et al.  Hierarchical Bayesian Nonparametric Models with Applications , 2008 .

[76]  Jim E. Griffin,et al.  Stick-breaking autoregressive processes , 2011 .

[77]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[78]  Mario Vento,et al.  Performance Evaluation of a People Tracking System on PETS2009 Database , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[79]  M. Steel,et al.  Time-dependent stick-breaking processes , 2009 .

[80]  P. Müller,et al.  A method for combining inference across related nonparametric Bayesian models , 2004 .

[81]  J. Kingman.  Random partitions in population genetics , 1978, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[82]  Yannick Boursier,et al.  Sparsity-driven people localization algorithm: Evaluation in crowded scenes environments , 2009, 2009 Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance.

[83]  Jim Pitman,et al.  Poisson–Dirichlet and GEM Invariant Distributions for Split-and-Merge Transformations of an Interval Partition , 2002, Combinatorics, Probability and Computing.

[84]  Jim Pitman,et al.  Probabilistic Bounds on the Coefficients of Polynomials with Only Real Zeros , 1997, J. Comb. Theory, Ser. A.

[85]  J. Griffin.  The Ornstein–Uhlenbeck Dirichlet process and other time-varying processes for Bayesian nonparametric inference , 2011 .

[86]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[87]  Yee Whye Teh,et al.  A Hierarchical Bayesian Language Model Based On Pitman-Yor Processes , 2006, ACL.