Approximate Inference in State-Space Models With Heavy-Tailed Noise

State-space models have been successfully applied across a wide range of problems ranging from system control to target tracking and autonomous navigation. Their ubiquity stems from their modeling flexibility, as well as the development of a battery of powerful algorithms for estimating the state variables. For multivariate models, the Gaussian noise assumption is predominant due its convenient computational properties. In some cases, anyhow, this assumption breaks down and no longer holds. We propose a novel approach to extending the applicability of this class of models to a wider range of noise distributions without losing the computational advantages of the associated algorithms. The estimation methods we develop parallel the Kalman filter and thus are readily implemented and inherit the same order of complexity. We derive all of the equations and algorithms from first principles. In order to validate the performance of our approach, we present specific instances of non-Gaussian state-space models and test their performance on experiments with synthetic and real data.

[1]  Joel M. Morris,et al.  The Kalman filter: A robust estimator for some classes of linear quadratic problems , 1976, IEEE Trans. Inf. Theory.

[2]  R. Muirhead Aspects of Multivariate Statistical Theory , 1982, Wiley Series in Probability and Statistics.

[3]  Michael A. West,et al.  Dynamic matrix-variate graphical models , 2007 .

[4]  David Barber,et al.  Expectation Correction for Smoothed Inference in Switching Linear Dynamical Systems , 2006, J. Mach. Learn. Res..

[5]  H. Sorenson,et al.  NONLINEAR FILTERING BY APPROXIMATION OF THE A POSTERIORI DENSITY , 1968 .

[6]  Tom Drummond,et al.  Student-tMixture Filter for Robust, Real-Time Visual Tracking , 2008, ECCV.

[7]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[8]  Wolfram Burgard,et al.  Improving Data Association in Vision-based SLAM , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[10]  Simo Särkkä,et al.  Recursive Noise Adaptive Kalman Filtering by Variational Bayesian Approximations , 2009, IEEE Transactions on Automatic Control.

[11]  Eduardo Mario Nebot,et al.  An outlier-robust Kalman filter , 2011, 2011 IEEE International Conference on Robotics and Automation.

[12]  H. Akaike A new look at the statistical model identification , 1974 .

[13]  Huaiyu Zhu On Information and Sufficiency , 1997 .

[14]  Geoffrey E. Hinton,et al.  Variational Learning for Switching State-Space Models , 2000, Neural Computation.

[15]  M. Genton,et al.  A skewed Kalman filter , 2005 .

[16]  P. Diaconis,et al.  Conjugate Priors for Exponential Families , 1979 .

[17]  A. Roverato Cholesky decomposition of a hyper inverse Wishart matrix , 2000 .

[18]  Paweł J. Szabłowski Elliptically contoured random variables and their application to the extension of the Kalman filter , 1990 .

[19]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[20]  R. E. Kalman,et al.  New Results in Linear Filtering and Prediction Theory , 1961 .

[21]  D. Ruppert Robust Statistics: The Approach Based on Influence Functions , 1987 .

[22]  Hugh Durrant-Whyte,et al.  Simultaneous localization and mapping (SLAM): part II , 2006 .

[23]  S. Haykin,et al.  Cubature Kalman Filters , 2009, IEEE Transactions on Automatic Control.

[24]  Guanrong Chen,et al.  Introduction to random signals and applied Kalman filtering, 2nd edn. Robert Grover Brown and Patrick Y. C. Hwang, Wiley, New York, 1992. ISBN 0‐471‐52573‐1, 512 pp., $62.95. , 1992 .

[25]  Jacques H. Dreze,et al.  Bayesian regression analysis using poly-t densities , 1977 .

[26]  G'erard Letac,et al.  Wishart distributions for decomposable graphs , 2007, 0708.2380.

[27]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[28]  Stefan Schaal,et al.  Automatic Outlier Detection: A Bayesian Approach , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[29]  F. J. Girón,et al.  Bayesian Kalman filtering with elliptically contoured errors , 1994 .

[30]  Zoubin Ghahramani,et al.  A Unifying Review of Linear Gaussian Models , 1999, Neural Computation.

[31]  Stefan Schaal,et al.  Learning an Outlier-Robust Kalman Filter , 2007, ECML.

[32]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[33]  A. O'Hagan A moment of indecision , 1981 .

[34]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[35]  M. Abramowitz,et al.  Handbook of Mathematical Functions With Formulas, Graphs and Mathematical Tables (National Bureau of Standards Applied Mathematics Series No. 55) , 1965 .

[36]  T. Kailath,et al.  New square-root smoothing algorithms , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[37]  P. Dooren,et al.  Numerical aspects of different Kalman filter implementations , 1986 .

[38]  Student,et al.  THE PROBABLE ERROR OF A MEAN , 1908 .

[39]  W. R. Buckland,et al.  Outliers in Statistical Data , 1979 .

[40]  David J. Fleet,et al.  Robust Online Appearance Models for Visual Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Fang Liu Bayesian Time Series: Analysis Methods Using Simulation-Based Computation , 2000 .

[42]  Ronald K. Pearson,et al.  Outliers in process modeling and identification , 2002, IEEE Trans. Control. Syst. Technol..

[43]  Tamar Frankel [The theory and the practice...]. , 2001, Tijdschrift voor diergeneeskunde.

[44]  竹安 数博,et al.  Time series analysis and its applications , 2007 .

[45]  J. Jensen Sur les fonctions convexes et les inégalités entre les valeurs moyennes , 1906 .

[46]  Petros G. Voulgaris,et al.  On optimal ℓ∞ to ℓ∞ filtering , 1995, Autom..

[47]  Hugh F. Durrant-Whyte,et al.  Simultaneous localization and mapping: part I , 2006, IEEE Robotics & Automation Magazine.

[48]  Hugh Durrant-Whyte,et al.  Simultaneous Localisation and Mapping ( SLAM ) : Part I The Essential Algorithms , 2006 .

[49]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[50]  H. Sorenson,et al.  Recursive bayesian estimation using gaussian sums , 1971 .

[51]  Milton Abramowitz,et al.  Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables , 1964 .

[52]  Stefan Schaal,et al.  A Kalman filter for robust outlier detection , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[53]  Matthew J. Beal Variational algorithms for approximate Bayesian inference , 2003 .

[54]  Thiagalingam Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation , 2001 .

[55]  R. E. Mortensen,et al.  Filtering for stochastic processes with applications to guidance , 1972 .

[56]  F. E. Grubbs Procedures for Detecting Outlying Observations in Samples , 1969 .

[57]  A. Dawid Some matrix-variate distribution theory: Notational considerations and a Bayesian application , 1981 .

[58]  Richard J. Meinhold,et al.  Robustification of Kalman Filter Models , 1989 .

[59]  J. Wishart THE GENERALISED PRODUCT MOMENT DISTRIBUTION IN SAMPLES FROM A NORMAL MULTIVARIATE POPULATION , 1928 .