Nonlinear Bayesian estimation: from Kalman filtering to a broader horizon

This article presents an up-to-date tutorial review of nonlinear Bayesian estimation. State estimation for nonlinear systems has been a challenge encountered in a wide range of engineering fields, attracting decades of research effort. To date, one of the most promising and popular approaches is to view and address the problem from a Bayesian probabilistic perspective, which enables estimation of the unknown state variables by tracking their probabilistic distribution or statistics U+0028 e.g., mean and covariance U+0029 conditioned on a system U+02BC s measurement data. This article offers a systematic introduction to the Bayesian state estimation framework and reviews various Kalman filtering U+0028 KF U+0029 techniques, progressively from the standard KF for linear systems to extended KF, unscented KF and ensemble KF for nonlinear systems. It also overviews other prominent or emerging Bayesian estimation methods including Gaussian filtering, Gaussian-sum filtering, particle filtering and moving horizon estimation and extends the discussion of state estimation to more complicated problems such as simultaneous state and parameter U+002F input estimation.

[1]  Aubrey B. Poore,et al.  Adaptive Gaussian Sum Filters for Space Surveillance , 2011, IEEE Transactions on Automatic Control.

[2]  Konrad Reif,et al.  Stochastic Stability of the Extended Kalman Filter With Intermittent Observations , 2010, IEEE Transactions on Automatic Control.

[3]  Laura Dovera,et al.  Multimodal ensemble Kalman filtering using Gaussian mixture models , 2011 .

[4]  Yang Cheng,et al.  Sparse Gauss-Hermite Quadrature Filter with Application to Spacecraft Attitude Estimation , 2011 .

[5]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[6]  J. Mandel,et al.  On the convergence of the ensemble Kalman filter , 2009, Applications of mathematics.

[7]  Fuli Wang,et al.  Hybrid Estimation of State and Input for Linear Discrete Time-varying Systems: A Game Theory Approach , 2008 .

[8]  A. Jazwinski.  Stochastic Processes and Filtering Theory , 1970 .

[9]  Wei Ping,et al.  A novel simplex unscented transform and filter , 2007, 2007 International Symposium on Communications and Information Technologies.

[10]  Ondrej Straka,et al.  Gaussian sum unscented Kalman filter with adaptive scaling parameters , 2011, 14th International Conference on Information Fusion.

[11]  G. Evensen.  Data Assimilation: The Ensemble Kalman Filter , 2006 .

[12]  Eric A. Wan,et al.  Dual Extended Kalman Filter Methods , 2002 .

[13]  James B. Rawlings,et al.  Efficient moving horizon estimation and nonlinear model predictive control , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[14]  Ondrej Straka,et al.  Unscented Kalman Filter: Aspects and Adaptive Setting of Scaling Parameter , 2012, IEEE Transactions on Automatic Control.

[15]  Huazhen Fang,et al.  State of charge estimation for lithium-ion batteries: An adaptive approach , 2014 .

[16]  R. D. Callafon,et al.  Ensemble-based simultaneous input and state estimation for nonlinear dynamic systems with application to wildfire data assimilation , 2017 .

[17]  Huazhen Fang,et al.  Smoothed estimation of unknown inputs and states in dynamic systems with application to oceanic flow field reconstruction , 2015 .

[18]  Barbara F. La Scala,et al.  An extended Kalman filter frequency tracker for high-noise environments , 1996, IEEE Trans. Signal Process..

[19]  Yu Peng,et al.  Review on cyber-physical systems , 2017, IEEE/CAA Journal of Automatica Sinica.

[20]  J. Mendel.  White-noise estimators for seismic data processing in oil exploration , 1977 .

[21]  Derrick Holliday,et al.  New natural observer applied to speed-sensorless DC servo and induction motors , 2004, IEEE Transactions on Industrial Electronics.

[22]  Peter D. Scott,et al.  Adaptive Gaussian Sum Filter for Nonlinear Bayesian Estimation , 2011, IEEE Transactions on Automatic Control.

[23]  Huazhen Fang,et al.  On the asymptotic stability of minimum-variance unbiased input and state estimation , 2012, Autom..

[24]  Lei Zhou,et al.  Speed sensorless state estimation for induction motors: A moving horizon approach , 2016, 2016 American Control Conference (ACC).

[25]  L. Fridman,et al.  Exact state estimation for linear systems with unknown inputs based on hierarchical super‐twisting algorithm , 2007 .

[26]  Huazhen Fang,et al.  Simultaneous input and state filtering: An ensemble approach , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[27]  F. W. Cathey,et al.  The iterated Kalman filter update as a Gauss-Newton method , 1993, IEEE Trans. Autom. Control..

[28]  John L. Crassidis.  Sigma-point Kalman filtering for integrated GPS and inertial navigation , 2006 .

[29]  G. P. Papavassilopoulos,et al.  Development and numerical investigation of new non-linear Kalman filter variants , 2011 .

[30]  Chien-Shu Hsieh,et al.  Robust two-stage Kalman filters for systems with unknown inputs , 2000, IEEE Trans. Autom. Control..

[31]  Henrique Marra Menegaz,et al.  A Systematization of the Unscented Kalman Filter Theory , 2015, IEEE Transactions on Automatic Control.

[32]  T. Bayes.  LII. An essay towards solving a problem in the doctrine of chances. By the late Rev. Mr. Bayes, F. R. S. communicated by Mr. Price, in a letter to John Canton, A. M. F. R. S , 1763, Philosophical Transactions of the Royal Society of London.

[33]  José Jaime Da Cruz,et al.  Complete offline tuning of the unscented Kalman filter , 2017, Autom..

[34]  Petar M. Djuric,et al.  Gaussian sum particle filtering , 2003, IEEE Trans. Signal Process..

[35]  David Q. Mayne,et al.  Constrained state estimation for nonlinear discrete-time systems: stability and moving horizon approximations , 2003, IEEE Trans. Autom. Control..

[36]  Li Liang-qun,et al.  The iterated extended Kalman particle filter , 2005, IEEE International Symposium on Communications and Information Technology, 2005. ISCIT 2005..

[37]  Soroosh Sorooshian,et al.  Dual state-parameter estimation of hydrological models using ensemble Kalman filter , 2005 .

[38]  Axel Barrau,et al.  The Invariant Extended Kalman Filter as a Stable Observer , 2014, IEEE Transactions on Automatic Control.

[39]  Matthias Morzfeld,et al.  Implicit particle filters for data assimilation , 2010, 1005.4002.

[40]  Cristiano Maria Verrelli,et al.  A global tracking control for speed-sensorless induction motors , 2004, Autom..

[41]  Fredrik Gustafsson,et al.  An efficient implementation of the second order extended Kalman filter , 2011, 14th International Conference on Information Fusion.

[42]  Elaine Martin,et al.  Particle filters for state and parameter estimation in batch processes , 2005 .

[43]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[44]  Simo Srkk,et al.  Bayesian Filtering and Smoothing , 2013 .

[45]  Chien-Shu Hsieh.  On the optimality of two‐stage Kalman filtering for systems with unknown inputs , 2010 .

[46]  Simo Särkkä,et al.  Bayesian Filtering and Smoothing , 2013, Institute of Mathematical Statistics textbooks.

[47]  Ming Xin,et al.  High-degree cubature Kalman filter , 2013, Autom..

[48]  Jean-Jacques E. Slotine,et al.  A Contraction Theory-Based Analysis of the Stability of the Deterministic Extended Kalman Filter , 2015, IEEE Transactions on Automatic Control.

[49]  Kazufumi Ito,et al.  Gaussian filters for nonlinear filtering problems , 2000, IEEE Trans. Autom. Control..

[50]  Xinping Guan,et al.  A comprehensive overview of cyber-physical systems: from perspective of feedback system , 2016, IEEE/CAA Journal of Automatica Sinica.

[51]  Chenglin Wen,et al.  Performance Analysis of the Kalman Filter With Mismatched Noise Covariances , 2016, IEEE Transactions on Automatic Control.

[52]  S. Lakshmivarahan,et al.  Ensemble Kalman filter , 2009, IEEE Control Systems.

[53]  Jan Mandel,et al.  Efficient Implementation of the Ensemble Kalman Filter Efficient Implementation of the Ensemble Kalman Filter , 2022 .

[54]  A. Chorin,et al.  Implicit sampling for particle filters , 2009, Proceedings of the National Academy of Sciences.

[55]  Miguel Ayala Botto,et al.  Simultaneous state and input estimation of hybrid systems with unknown inputs , 2006, Autom..

[56]  Nando de Freitas,et al.  The Unscented Particle Filter , 2000, NIPS.

[57]  Bart De Moor,et al.  Unbiased minimum-variance input and state estimation for linear discrete-time systems with direct feedthrough , 2007, Autom..

[58]  Carlo Rossi,et al.  Speed Sensorless Control of Induction Motors Based on a Reduced-Order Adaptive Observer , 2007, IEEE Transactions on Control Systems Technology.

[59]  Martin J. Corless,et al.  State and Input Estimation for a Class of Uncertain Systems , 1998, Autom..

[60]  Huazhen Fang,et al.  Adaptive estimation of state of charge for lithium-ion batteries , 2013, 2013 American Control Conference.

[61]  P. Houtekamer,et al.  Data Assimilation Using an Ensemble Kalman Filter Technique , 1998 .

[62]  R. V. Jategaonkar,et al.  Aerodynamic parameter estimation from flight data applying extended and unscented Kalman filter , 2006 .

[63]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[64]  Josep M. Guerrero,et al.  Industrial Applications of the Kalman Filter: A Review , 2013, IEEE Transactions on Industrial Electronics.

[65]  G. Evensen.  Sequential data assimilation with a nonlinear quasi‐geostrophic model using Monte Carlo methods to forecast error statistics , 1994 .

[66]  Senia,et al.  A new unscented Kalman filter with higher order moment-matching , 2010 .

[67]  G. Evensen,et al.  Assimilation of Geosat altimeter data for the Agulhas current using the ensemble Kalman filter with , 1996 .

[68]  Lei Zhou,et al.  High gain observer for speed-sensorless motor drives: Algorithm and experiments , 2016, 2016 IEEE International Conference on Advanced Intelligent Mechatronics (AIM).

[69]  Jingang Yi,et al.  A new algorithm for simultaneous input and state estimation , 2008, 2008 American Control Conference.

[70]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[71]  Hieu Minh Trinh,et al.  State and input simultaneous estimation for a class of nonlinear systems , 2004, Autom..

[72]  Bart De Moor,et al.  Unbiased minimum-variance input and state estimation for linear discrete-time systems , 2007, Autom..

[73]  A. Dale.  Bayes or Laplace? An examination of the origin and early applications of Bayes' theorem , 1982, Archive for History of Exact Sciences.

[74]  Rudolph van der Merwe,et al.  Sigma-point kalman filters for probabilistic inference in dynamic state-space models , 2004 .

[75]  Simon J. Julier,et al.  The spherical simplex unscented transformation , 2003, Proceedings of the 2003 American Control Conference, 2003..

[76]  Jingang Yi,et al.  On stable simultaneous input and state estimation for discrete‐time linear systems , 2011 .

[77]  T. Bayes.  An essay towards solving a problem in the doctrine of chances , 2003 .

[78]  Jianyu Yang,et al.  A Computationally Efficient Particle Filter for Multitarget Tracking Using an Independence Approximation , 2013, IEEE Transactions on Signal Processing.

[79]  S. Haykin,et al.  Cubature Kalman Filters , 2009, IEEE Transactions on Automatic Control.

[80]  Huazhen Fang,et al.  Simultaneous input and state estimation for nonlinear systems with applications to flow field estimation , 2013, Autom..

[81]  H.F. Durrant-Whyte,et al.  A new approach for filtering nonlinear systems , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[82]  Arnaud Doucet,et al.  Sequential Monte Carlo Methods to Train Neural Network Models , 2000, Neural Computation.

[83]  A M Vann,et al.  Dealing with Data Overload , 1993 .

[84]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[85]  A. Krener.  The Convergence of the Extended Kalman Filter , 2002, math/0212255.

[86]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[87]  Jouni Hartikainen,et al.  On the relation between Gaussian process quadratures and sigma-point methods , 2015, 1504.05994.

[88]  Rudolph van der Merwe,et al.  Dual Estimation and the Unscented Transformation , 1999, NIPS.

[89]  J. Beck,et al.  Bayesian State and Parameter Estimation of Uncertain Dynamical Systems , 2006 .

[90]  T. Floquet,et al.  On Sliding Mode Observers for Systems with Unknown Inputs , 2006, International Workshop on Variable Structure Systems, 2006. VSS'06..

[91]  M. Boutayeb,et al.  Convergence analysis of the extended Kalman filter used as an observer for nonlinear deterministic discrete-time systems , 1997, IEEE Trans. Autom. Control..

[92]  Jeffrey K. Uhlmann,et al.  Unscented filtering and nonlinear estimation , 2004, Proceedings of the IEEE.

[93]  Hans Joachim Ferreau,et al.  Efficient Numerical Methods for Nonlinear MPC and Moving Horizon Estimation , 2009 .

[94]  D Malakoff,et al.  Bayes Offers a 'New' Way to Make Sense of Numbers , 1999, Science.

[95]  Y. Ho,et al.  A Bayesian approach to problems in stochastic estimation and control , 1964 .

[96]  Rudolph van der Merwe,et al.  The square-root unscented Kalman filter for state and parameter-estimation , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[97]  Hisashi Tanizaki,et al.  Nonlinear Filters: Estimation and Applications , 1993 .

[98]  Arnaud Doucet,et al.  Convergence of Sequential Monte Carlo Methods , 2007 .

[99]  James V. Candy,et al.  Bayesian Signal Processing: Classical, Modern and Particle Filtering Methods , 2009 .

[100]  M. C. VanDyke,et al.  UNSCENTED KALMAN FILTERING FOR SPACECRAFT ATTITUDE STATE AND PARAMETER ESTIMATION , 2004 .