Mixture-based extension of the AR model and its recursive Bayesian identification

An extension of the AutoRegressive (AR) model is studied, which allows transformations and distortions on the regressor to be handled. Many important signal processing problems are amenable to this Extended AR (i.e., EAR) model. It is shown that Bayesian identification and prediction of the EAR model can be performed recursively, in common with the AR model itself. The EAR model does, however, require that the transformation be known. When it is unknown, the associated transformation space is represented by a finite set of candidates. What follows is a Mixture-based EAR model, i.e., the MEAR model. An approximate identification algorithm for MEAR is developed, using a restricted Variational Bayes (VB) method. This restores the elegant recursive update of sufficient statistics. The MEAR model is applied to the robust identification of AR processes corrupted by outliers and burst noise, respectively, and to click removal for speech.

[1]  Maciej Niedzwiecki,et al.  Adaptive scheme for elimination of broadband noise and impulsive disturbances from AR and ARMA signals , 1996, IEEE Trans. Signal Process..

[2]  M. Opper,et al.  Advanced mean field methods: theory and practice , 2001 .

[3]  R. Kulhavý.  Restricted exponential forgetting in real-time identification , 1985, at - Automatisierungstechnik.

[4]  B. O. Koopman.  On distributions admitting a sufficient statistic , 1936 .

[5]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[6]  J. J. Rajan,et al.  Bayesian approach to parameter estimation and interpolation of time-varying autoregressive processes using the Gibbs sampler , 1997 .

[7]  Chun. Loo,et al.  BAYESIAN APPROACH TO SYSTEM IDENTIFICATION , 1981 .

[8]  D. Pfeffermann,et al.  Small area estimation , 2011 .

[9]  Anthony Quinn.  Regularized Signal Identification Using Bayesian Techniques , 1998 .

[10]  Irene A. Stegun,et al.  Handbook of Mathematical Functions. , 1966 .

[11]  Hagai Attias,et al.  A Variational Bayesian Framework for Graphical Models , 1999 .

[12]  G. Bierman.  Factorization methods for discrete sequential estimation , 1977 .

[13]  Tatiana V. Guy,et al.  Robust estimation of autoregressive processes using a mixture-based filter-bank , 2005, Syst. Control. Lett..

[14]  J. Makhoul,et al.  Linear prediction: A tutorial review , 1975, Proceedings of the IEEE.

[15]  G. Kitagawa,et al.  A smoothness priors time-varying AR coefficient modeling of nonstationary covariance time series , 1985, IEEE Transactions on Automatic Control.

[16]  William D. Penny,et al.  Variational Bayes for generalized autoregressive models , 2002, IEEE Trans. Signal Process..

[17]  Dror G. Feitelson,et al.  Self-Tuning Systems , 1999, IEEE Softw..

[18]  Mary R. Hardy,et al.  A Regime-Switching Model of Long-Term Stock Returns , 2001 .

[19]  P. Davies,et al.  Kendall's Advanced Theory of Statistics. Volume 1. Distribution Theory , 1988 .

[20]  Ivan Nagy,et al.  Probabilistic advisory systems for data‐intensive applications , 2003 .

[21]  S. Thomas Alexander,et al.  Adaptive Signal Processing , 1986, Texts and Monographs in Computer Science.

[22]  George V. Moustakides,et al.  Locally optimum adaptive signal processing algorithms , 1998, IEEE Trans. Signal Process..

[23]  Tatiana V. Guy,et al.  Mixture‐based adaptive probabilistic control , 2003 .

[24]  D. Mayne,et al.  Design issues in adaptive control , 1988 .

[25]  M.G. Bellanger,et al.  Digital processing of speech signals , 1980, Proceedings of the IEEE.

[26]  Y. Bar-Shalom,et al.  Multiple-model estimation with variable structure , 1996, IEEE Trans. Autom. Control..

[27]  R. Kulhavý,et al.  On a general concept of forgetting , 1993 .

[28]  P. Kumar,et al.  Theory and practice of recursive identification , 1985, IEEE Transactions on Automatic Control.

[29]  A. Quinn,et al.  Bayesian estimation of non-stationary AR model parameters via an unknown forgetting factor , 2004, 3rd IEEE Signal Processing Education Workshop. 2004 IEEE 11th Digital Signal Processing Workshop, 2004..

[30]  Rudolf Kulhavý,et al.  On duality of regularized exponential and linear forgetting , 1996, Autom..

[31]  Václav Peterka.  Real-time parameter estimation and output prediction for ARMA-type system models , 1981, Kybernetika.

[32]  Miroslav Kárný,et al.  Estimation and prediction with ARMMAX model: a mixture of ARMAX models with common ARX part , 2003 .

[33]  M. West,et al.  Dynamic Generalized Linear Models and Bayesian Forecasting , 1985 .