Bayesian Inference for Linear Dynamic Models With Dirichlet Process Mixtures

Using Kalman techniques, it is possible to perform optimal estimation in linear Gaussian state-space models. Here, we address the case where the noise probability density functions are of unknown functional form. A flexible Bayesian nonparametric noise model based on Dirichlet process mixtures is introduced. Efficient Markov chain Monte Carlo and sequential Monte Carlo methods are then developed to perform optimal batch and sequential estimation in such contexts. The algorithms are applied to blind deconvolution and change point detection. Experimental results on synthetic and real data demonstrate the efficiency of this approach in various contexts.

[1]  K. Fu,et al.  On state estimation in switching environments , 1968 .

[2]  K. Ito,et al.  On State Estimation in Switching Environments , 1970 .

[3]  R. Mehra.  On the identification of variances and adaptive Kalman filtering , 1970 .

[4]  D. Blackwell,et al.  Ferguson Distributions Via Polya Urn Schemes , 1973 .

[5]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[6]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[7]  B. Tapley,et al.  Adaptive sequential estimation with unknown noise statistics , 1976 .

[8]  Hiromitsu Kumamoto,et al.  Random sampling approach to state estimation in switching environments , 1977, Autom..

[9]  R. Maine,et al.  Formulation and implementation of a practical algorithm for parameter estimation with process and measurement noise , 1980 .

[10]  K. Gordon,et al.  Modeling and Monitoring Biomedical Time Series , 1990 .

[11]  J. Sethuraman.  A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[12]  M. West,et al.  Hyperparameter estimation in Dirichlet process mixture models , 1992 .

[13]  N. Shephard.  Partial non-Gaussian state space , 1994 .

[14]  Petros G. Voulgaris,et al.  On optimal ℓ∞ to ℓ∞ filtering , 1995, Autom..

[15]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[16]  R. Kohn,et al.  Markov chain Monte Carlo in conditionally Gaussian state space models , 1996 .

[17]  A. Raftery,et al.  A note on the Dirichlet process prior in Bayesian nonparametric inference with partial exchangeability , 1997 .

[18]  J. L. Maryak,et al.  Use of the Kalman filter for inference in state-space models with unknown noise distributions , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[19]  Patrick Duvaut,et al.  Bayesian estimation of state-space models applied to deconvolution of Bernoulli - Gaussian processes , 1997, Signal Process..

[20]  Jun S. Liu,et al.  Sequential importance sampling for nonparametric Bayes models: The next generation , 1999 .

[21]  Purushottam W. Laud,et al.  Bayesian Nonparametric Inference for Random Distributions and Related Functions , 1999 .

[22]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[23]  A. Pievatolo,et al.  Analysing the interevent time distribution to identify seismicity phases: a Bayesian nonparametric approach to the multiple‐changepoint problem , 2000 .

[24]  Radford M. Neal.  Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[25]  M. Escobar,et al.  Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[26]  Steven N. MacEachern,et al.  Efficient MCMC Schemes for Robust Model Extensions Using Encompassing Dirichlet Process Mixture Models , 2000 .

[27]  Christophe Andrieu,et al.  Iterative algorithms for state estimation of jump Markov linear systems , 2001, IEEE Trans. Signal Process..

[28]  W. Gilks,et al.  Following a moving target—Monte Carlo inference for dynamic Bayesian models , 2001 .

[29]  Arnaud Doucet,et al.  Particle filters for state estimation of jump Markov linear systems , 2001, IEEE Trans. Signal Process..

[30]  Siem Jan Koopman,et al.  A simple and efficient simulation smoother for state space time series analysis , 2002 .

[31]  Michael A. West,et al.  Combined Parameter and State Estimation in Simulation-Based Filtering , 2001, Sequential Monte Carlo Methods in Practice.

[32]  P. Green,et al.  Modelling Heterogeneity With and Without the Dirichlet Process , 2001 .

[33]  Mario Medvedovic,et al.  Bayesian infinite mixture model based clustering of gene expression profiles , 2002, Bioinform..

[34]  M. Steel,et al.  Semiparametric Bayesian Inference for Stochastic Frontier Models , 2004 .

[35]  Christophe Andrieu,et al.  Efficient particle filtering for jump Markov systems. Application to time-varying autoregressions , 2003, IEEE Trans. Signal Process..

[36]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[37]  A. Doucet,et al.  Parameter estimation in general state-space models using particle methods , 2003 .

[38]  Fernando A. Quintana,et al.  Nonparametric Bayesian data analysis , 2004 .

[39]  J. L. Maryak,et al.  Use of the Kalman filter for inference in state-space models with unknown noise distributions , 1996, IEEE Transactions on Automatic Control.

[40]  Paul Fearnhead,et al.  Particle filters for mixture models with an unknown number of components , 2004, Stat. Comput..

[41]  Marco J. Lombardi,et al.  On-line Bayesian Estimation of Signals in Symmetric α-Stable Noise , 2004 .

[42]  P. Müller,et al.  A Bayesian mixture model for differential gene expression , 2005 .

[43]  Arnaud Doucet,et al.  Particle methods for optimal filter derivative: application to parameter estimation , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[44]  Volkan Cevher,et al.  General direction-of-arrival tracking with acoustic nodes , 2005, IEEE Transactions on Signal Processing.

[45]  Simon J. Godsill,et al.  On-line Bayesian estimation of signals in symmetric /spl alpha/-stable noise , 2006, IEEE Transactions on Signal Processing.

[46]  Michael A. West,et al.  Hierarchical priors and mixture models, with applications in regression and density estimation , 2006 .

[47]  Arnaud Doucet,et al.  Bayesian Inference for Dynamic Models with Dirichlet Process Mixtures , 2006, 2006 9th International Conference on Information Fusion.

[48]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.