Joint Modeling of Multiple Related Time Series via the Beta Process

We propose a Bayesian nonparametric approach to the problem of jointly modeling multiple related time series. Our approach is based on the discovery of a set of latent, shared dynamical behaviors. Using a beta process prior, the size of the set and the sharing pattern are both inferred from data. We develop efficient Markov chain Monte Carlo methods based on the Indian buffet process representation of the predictive distribution of the beta process, without relying on a truncated model. In particular, our approach uses the sum-product algorithm to efficiently compute Metropolis-Hastings acceptance probabilities, and explores new dynamical behaviors via birth and death proposals. We examine the benefits of our proposed feature-based model on several synthetic datasets, and also demonstrate promising results on unsupervised segmentation of visual motion capture data.

[1]  J. Kingman,et al.  Completely random measures. , 1967 .

[2]  M. West,et al.  Bayesian forecasting and dynamic models , 1989 .

[3]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[4]  N. Hjort.  Nonparametric Bayes Estimators Based on Beta Processes in Models for Life History Data , 1990 .

[5]  M. Aoki,et al.  State space modeling of multiple time series , 1991 .

[6]  C. Hwang,et al.  Convergence rates of the Gibbs sampler, the Metropolis algorithm and other single-site updating dynamics , 1993 .

[7]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[8]  Jun S. Liu.  Peskun's theorem and a modified discrete-state Gibbs sampler , 1996 .

[9]  Vladimir Pavlovic,et al.  A dynamic Bayesian network approach to figure tracking using learned dynamic models , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[10]  Yongdai Kim.  NONPARAMETRIC BAYESIAN ESTIMATORS FOR COUNTING PROCESSES , 1999 .

[11]  S. MacEachern.  Decision Theoretic Aspects of Dependent Nonparametric Processes , 2000 .

[12]  Vladimir Pavlovic,et al.  Learning Switching Linear Models of Human Motion , 2000, NIPS.

[13]  George M. Church,et al.  Aligning gene expression time series with time warping algorithms , 2001, Bioinform..

[14]  Carl E. Rasmussen,et al.  Factorial Hidden Markov Models , 1997 .

[15]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[16]  Vladimir Pavlovic,et al.  Discovering clusters in motion time-series data , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[17]  Jernej Barbic,et al.  Segmenting Motion Capture Data into Distinct Behaviors , 2004, Graphics Interface.

[18]  Michael I. Jordan,et al.  Factorial Hidden Markov Models , 1995, Machine Learning.

[19]  Kari Pulli,et al.  Style translation for human motion , 2005, SIGGRAPH 2005.

[20]  Kevin Duh,et al.  Jointly Labeling Multiple Sequences: A Factorial HMM Approach , 2005, ACL.

[21]  Thomas L. Griffiths,et al.  Infinite latent feature models and the Indian buffet process , 2005, NIPS.

[22]  Ajay Jasra,et al.  Markov Chain Monte Carlo Methods and the Label Switching Problem in Bayesian Mixture Modeling , 2005 .

[23]  J. E. Griffin,et al.  Order-Based Dependent Dirichlet Processes , 2006 .

[24]  Carl E. Rasmussen,et al.  A choice model with infinitely many latent features , 2006, ICML.

[25]  Radford M. Neal,et al.  Bayesian Detection of Infrequent Differences in Sets of Time Series with Shared Structure , 2006, NIPS.

[26]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[27]  R. Altman.  Mixed Hidden Markov Models , 2007 .

[28]  B. Schölkopf,et al.  Modeling Human Motion Using Binary Latent Variables , 2007 .

[29]  B. Schölkopf,et al.  Modeling Dyadic Data with Binary Latent Factors , 2007 .

[30]  Michael I. Jordan,et al.  Hierarchical Beta Processes and the Indian Buffet Process , 2007, AISTATS.

[31]  Lawrence Carin,et al.  Music Analysis Using Hidden Markov Mixture Models , 2007, IEEE Transactions on Signal Processing.

[32]  David J. Fleet,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE Gaussian Process Dynamical Model , 2007 .

[33]  Yee Whye Teh,et al.  The Infinite Factorial Hidden Markov Model , 2008, NIPS.

[34]  Michael I. Jordan,et al.  Sharing Features among Dynamical Systems with Beta Processes , 2009, NIPS.

[35]  D. Dunson.  Nonparametric Bayes local partition models for random effects. , 2009, Biometrika.

[36]  Wolfgang P. Lehrach,et al.  Segmenting bacterial and viral DNA sequence alignments with a trans‐dimensional phylogenetic factorial hidden Markov model , 2009 .

[37]  D. Dunson.  MULTIVARIATE KERNEL PARTITION PROCESS MIXTURES. , 2010, Statistica Sinica.

[38]  Michael I. Jordan,et al.  Bayesian Nonparametric Methods for Learning Markov Switching Processes , 2010, IEEE Signal Processing Magazine.

[39]  Suchi Saria,et al.  Discovering shared and individual latent structure in multiple time series , 2010, ArXiv.

[40]  M. R. Leadbetter.  Poisson Processes , 2011, International Encyclopedia of Statistical Science.

[41]  Michael I. Jordan,et al.  A Sticky HDP-HMM With Application to Speaker Diarization , 2009, 0905.2592.

[42]  Michael I. Jordan,et al.  Bayesian Nonparametric Inference of Switching Dynamic Linear Models , 2010, IEEE Transactions on Signal Processing.