Mixed Membership Models for Time Series

In this article we discuss some of the consequences of the mixed membership perspective on time series analysis. In its most abstract form, a mixed membership model aims to associate an individual entity with some set of attributes based on a collection of observed data. Although much of the literature on mixed membership models considers the setting in which exchangeable collections of data are associated with each member of a set of entities, it is equally natural to consider problems in which an entire time series is viewed as an entity and the goal is to characterize the time series in terms of a set of underlying dynamic attributes or "dynamic regimes". Indeed, this perspective is already present in the classical hidden Markov model, where the dynamic regimes are referred to as "states", and the collection of states realized in a sample path of the underlying process can be viewed as a mixed membership characterization of the observed time series. Our goal here is to review some of the richer modeling possibilities for time series that are provided by recent developments in the mixed membership framework.

[1]  Peter Müller,et al.  Bayesian Models for Non‐linear Autoregressions , 1997 .

[2]  Michael I. Jordan,et al.  An HDP-HMM for systems with state persistence , 2008, ICML '08.

[3]  Thomas L. Griffiths,et al.  Infinite latent feature models and the Indian buffet process , 2005, NIPS.

[4]  Yongdai Kim.  NONPARAMETRIC BAYESIAN ESTIMATORS FOR COUNTING PROCESSES , 1999 .

[5]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[6]  Michael I. Jordan,et al.  Hierarchical Beta Processes and the Indian Buffet Process , 2007, AISTATS.

[7]  Lawrence Carin,et al.  A Stick-Breaking Construction of the Beta Process , 2010, ICML.

[8]  Yee Whye Teh,et al.  Infinite Hierarchical Hidden Markov Models , 2009, AISTATS.

[9]  P. McCullagh.  Partition models , 2015 .

[10]  Lawrence Carin,et al.  Music Analysis Using Hidden Markov Mixture Models , 2007, IEEE Transactions on Signal Processing.

[11]  J. Hartigan,et al.  Product Partition Models for Change Point Problems , 1992 .

[12]  John D. Lafferty,et al.  Dynamic topic models , 2006, ICML.

[13]  Michael I. Jordan,et al.  A Sticky HDP-HMM With Application to Speaker Diarization , 2009, 0905.2592.

[14]  Jim E. Griffin,et al.  Stick-breaking autoregressive processes , 2011 .

[15]  Chang‐Jin Kim,et al.  Dynamic linear models with Markov-switching , 1994 .

[16]  Robert Kohn,et al.  Bayesian Mixtures of Autoregressive Models , 2011 .

[17]  Yee Whye Teh,et al.  Beam sampling for the infinite hidden Markov model , 2008, ICML '08.

[18]  Marina Vannucci,et al.  A DIRICHLET PROCESS MIXTURE OF HIDDEN MARKOV MODELS FOR PROTEIN STRUCTURE PREDICTION. , 2010, The annals of applied statistics.

[19]  Paul Fearnhead,et al.  Exact and efficient Bayesian inference for multiple changepoint problems , 2006, Stat. Comput..

[20]  N. Hjort.  Nonparametric Bayes Estimators Based on Beta Processes in Models for Life History Data , 1990 .

[21]  Michael I. Jordan,et al.  Sharing Features among Dynamical Systems with Beta Processes , 2009, NIPS.

[22]  P. Donnelly,et al.  Association mapping in structured populations. , 2000, American journal of human genetics.

[23]  Chong Wang,et al.  The IBP Compound Dirichlet Process and its Application to Focused Topic Modeling , 2010, ICML.

[24]  S. Roweis,et al.  Time-Varying Topic Models using Dependent Dirichlet Processes , 2005 .

[25]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[26]  Ramsés H. Mena,et al.  Geometric stick-breaking processes for continuous-time Bayesian nonparametric modeling , 2011 .

[27]  Kevin P. Murphy,et al.  Modeling changing dependency structure in multivariate time series , 2007, ICML '07.

[28]  Paul Damien,et al.  Predictive Macro-Finance With Dynamic Partition Models , 2011 .

[29]  S. MacEachern.  Decision Theoretic Aspects of Dependent Nonparametric Processes , 2000 .

[30]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[31]  J. Pitman.  Combinatorial Stochastic Processes , 2006 .

[32]  Matt Taddy.  Autoregressive Mixture Models for Dynamic Spatial Poisson Processes: Application to Tracking Intensity of Violent Crime , 2010 .

[33]  Ardavan Saeedi,et al.  Priors over Recurrent Continuous Time Processes , 2011, NIPS.

[34]  Leslie M. Collins,et al.  Variational Bayesian Learning for Mixture Autoregressive Models With Uncertain-Order , 2011, IEEE Transactions on Signal Processing.

[35]  Chong Wang,et al.  Continuous Time Dynamic Topic Models , 2008, UAI.

[36]  Michael I. Jordan,et al.  Bayesian Nonparametric Inference of Switching Dynamic Linear Models , 2010, IEEE Transactions on Signal Processing.

[37]  John W. Lau,et al.  Bayesian mixture of autoregressive models , 2008, Comput. Stat. Data Anal..

[38]  C. Yau,et al.  Bayesian non‐parametric hidden Markov models with applications in genomics , 2011 .

[39]  Carl E. Rasmussen,et al.  Factorial Hidden Markov Models , 1997 .

[40]  Matt Taddy,et al.  Markov switching Dirichlet process mixture regression , 2009 .

[41]  J. Sethuraman.  A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .