Exact and efficient Bayesian inference for multiple changepoint problems

We demonstrate how to perform direct simulation from the posterior distribution of a class of multiple changepoint models where the number of changepoints is unknown. The class of models assumes independence between the posterior distribution of the parameters associated with segments of data between successive changepoints. This approach is based on the use of recursions, and is related to work on product partition models. The computational complexity of the approach is quadratic in the number of observations, but an approximate version, which introduces negligible error, and whose computational cost is roughly linear in the number of observations, is also possible. Our approach can be useful, for example within an MCMC algorithm, even when the independence assumptions do not hold. We demonstrate our approach on coal-mining disaster data and on well-log data. Our method can cope with a range of models, and exact simulation from the posterior distribution is possible in a matter of minutes.

[1]  Arjun K. Gupta,et al.  Testing and Locating Variance Changepoints with Application to Stock Prices , 1997 .

[2]  David Bruce Wilson,et al.  Exact sampling with coupled Markov chains and applications to statistical mechanics , 1996, Random Struct. Algorithms.

[3]  A. Raftery,et al.  Bayesian analysis of a Poisson process with a change-point , 1986 .

[4]  Paul Fearnhead,et al.  Exact Bayesian curve fitting and signal segmentation , 2005, IEEE Transactions on Signal Processing.

[5]  Marc Lavielle,et al.  An application of MCMC methods for the multiple change-points problem , 2001, Signal Process..

[6]  H. Bergman,et al.  Detection of onset of neuronal activity by allowing for heterogeneity in the change points , 2002, Journal of Neuroscience Methods.

[7]  S. Chib,et al.  Bayes inference via Gibbs sampling of autoregressive time series subject to Markov mean and variance shifts , 1993 .

[8]  P. Fearnhead,et al.  Exact filtering for partially observed continuous time models , 2004 .

[9]  Paul Fearnhead Direct simulation for discrete mixture distributions , 2005, Stat. Comput..

[10]  P. Fearnhead,et al.  Improved particle filter for nonlinear problems , 1999 .

[11]  D. Stephens Bayesian Retrospective Multiple‐Changepoint Identification , 1994 .

[12]  R. Jarrett A note on the intervals between coal-mining disasters , 1979 .

[13]  P. Fearnhead,et al.  On‐line inference for multiple changepoint problems , 2007 .

[14]  J. Hartigan,et al.  A Bayesian Analysis for Change Point Problems , 1993 .

[15]  P. Fearnhead,et al.  Efficient Online Inference for Multiple Changepoint Problems , 2006, 2006 IEEE Nonlinear Statistical Signal Processing Workshop.

[16]  G. Roberts,et al.  Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions , 2003 .

[17]  P. Fearnhead,et al.  An improved particle filter for non-linear problems , 1999 .

[18]  Jun S. Liu,et al.  Bayesian inference on biopolymer models , 1999, Bioinform..

[19]  P. Fearnhead,et al.  On‐line inference for hidden Markov models via particle filters , 2003 .

[20]  S. Chib Marginal Likelihood from the Gibbs Output , 1995 .

[21]  Yi-Ching Yao Estimation of a Noisy Discrete-Time Step Function: Bayes and Empirical Bayes Approaches , 1984 .

[22]  Robert Lund,et al.  Detection of Undocumented Changepoints: A Revision of the Two-Phase Regression Model , 2002 .

[23]  Adrian F. M. Smith,et al.  Hierarchical Bayesian Analysis of Changepoint Problems , 1992 .

[24]  P. Green,et al.  Trans-dimensional Markov chain Monte Carlo , 2000 .

[25]  S. Chib Calculating posterior distributions and modal estimates in Markov mixture models , 1996 .

[26]  Lynn Kuo,et al.  Bayesian Binary Segmentation Procedure for a Poisson Process With Multiple Changepoints , 2001 .

[27]  P. Green,et al.  Boundary detection through dynamic polygons , 1998 .

[28]  H. Müller,et al.  Multiple changepoint fitting via quasilikelihood, with application to DNA sequence segmentation , 2000 .

[29]  P. Green Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[30]  S. Harkema,et al.  A Bayesian change-point analysis of electromyographic data: detecting muscle activation patterns and associated applications. , 2003, Biostatistics.

[31]  Keith J. Worsley,et al.  On the Likelihood Ratio Test for a Shift in Location of Normal Populations , 1979 .

[32]  S. Chib Estimation and comparison of multiple change-point models , 1998 .

[33]  H. Müller,et al.  Statistical methods for DNA sequence segmentation , 1998 .

[34]  S. L. Scott Bayesian Methods for Hidden Markov Models , 2002 .

[35]  K OrJ Numerical Bayesian methods applied to signal processing , 1996 .

[36]  J. Hartigan,et al.  Product Partition Models for Change Point Problems , 1992 .

[37]  Christophe Andrieu,et al.  Bayesian curve fitting using MCMC with applications to signal segmentation , 2002, IEEE Trans. Signal Process..

[38]  Tim Hesterberg,et al.  Monte Carlo Strategies in Scientific Computing , 2002, Technometrics.