Empirical Bayesian Change Point Detection

This paper explores a Bayesian method for the detection of sudden changes in the generative parameters of a data series. The problem is phrased as a hidden Markov model, where change point locations correspond to unobserved states, which grow in number with the number of observations. Our interest lies in the marginal change point posterior density. Rather than optimize a likelihood function of model parameters, we adapt the Baum-Welch algorithm to maximize a bound on the log marginal likelihood with respect to prior hyperparameters. This empirical Bayesian approach allows scale-invariance, and can be viewed as an expectation maximization algorithm for hyperparameter optimization in conjugate exponential models with latent variables. The expectation and maximization steps make respective use of variational and concave-convex inner loops. A judicious choice of change point prior allows for fast recursive computations on a graphical model. Results are shown on a number of real-world data sets.

[1]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[2]  L. Baum,et al.  An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology , 1967 .

[3]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[4]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[5]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[6]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[7]  David J. C. MacKay,et al.  The Evidence Framework Applied to Classification Networks , 1992, Neural Computation.

[8]  J. Hartigan,et al.  Product Partition Models for Change Point Problems , 1992 .

[9]  A. Erhan Mergen,et al.  IMPROVING THE PERFORMANCE OF THE T2 CONTROL CHART , 1993 .

[10]  J. Hartigan,et al.  A Bayesian Analysis for Change Point Problems , 1993 .

[11]  T. Lai.  Sequential changepoint detection in quality control and dynamical systems , 1995 .

[12]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[13]  Bradley P. Carlin,et al.  BAYES AND EMPIRICAL BAYES METHODS FOR DATA ANALYSIS , 1996, Stat. Comput..

[14]  S. Chib.  Estimation and comparison of multiple change-point models , 1998 .

[15]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[16]  Hagai Attias,et al.  A Variational Bayesian Framework for Graphical Models , 1999 .

[17]  Bradley P. Carlin,et al.  BAYES AND EMPIRICAL BAYES METHODS FOR DATA ANALYSIS , 1996, Stat. Comput..

[18]  William H. Woodall,et al.  Change-point detection of mean vector or covariance matrix shifts using multivariate individual observations , 2000 .

[19]  Nuno Vasconcelos,et al.  Empirical Bayesian Motion Segmentation , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Adrian Corduneanu,et al.  Variational Bayesian Model Selection for Mixture Distributions , 2001 .

[21]  Rosangela Helena Loschi,et al.  Applying the Product Partition Model to the Identification of Multiple Change Points , 2002, Adv. Complex Syst..

[22]  Alan L. Yuille,et al.  CCCP Algorithms to Minimize the Bethe and Kikuchi Free Energies: Convergent Alternatives to Belief Propagation , 2002, Neural Computation.

[23]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[24]  Charles M. Bishop,et al.  Variational Message Passing , 2005, J. Mach. Learn. Res..

[25]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[26]  Ulrich Paquet.  Bayesian inference for latent variable models , 2007 .

[27]  Ryan P. Adams,et al.  Bayesian Online Changepoint Detection , 2007, 0710.3742.