Intrinsically Bayesian Robust Kalman Filter: An Innovation Process Approach

In many contemporary engineering problems, model uncertainty is inherent because accurate system identification is virtually impossible owing to system complexity or lack of data on account of availability, time, or cost. The situation can be treated by assuming that the true model belongs to an uncertainty class of models. In this context, an intrinsically Bayesian robust (IBR) filter is one that is optimal relative to the cost function (in the classical sense) and the prior distribution over the uncertainty class (in the Bayesian sense). IBR filters have previously been found for both Wiener and granulometric morphological filtering. In this paper, we derive the IBR Kalman filter that performs optimally relative to an uncertainty class of state-space models. Introducing the notion of Bayesian innovation process and the Bayesian orthogonality principle, we show how the problem of designing an IBR Kalman filter can be reduced to a recursive system similar to the classical Kalman recursive equations, except with “effective” counterparts, such as the effective Kalman gain matrix. After deriving the recursive IBR Kalman equations for discrete time, we use the limiting method to obtain the IBR Kalman–Bucy equations for continuous time. Finally, we demonstrate the utility of the proposed framework for two real world problems: sensor networks and gene regulatory network inference.

[1]  E.R. Dougherty,et al.  Research issues in genomic signal processing , 2005, IEEE Signal Processing Magazine.

[2]  Pramod K. Varshney,et al.  Theory of the Stochastic Resonance Effect in Signal Detection—Part II: Variable Detectors , 2007, IEEE Transactions on Signal Processing.

[3]  Tong Zhou,et al.  Sensitivity Penalization Based Robust State Estimation for Uncertain Linear Systems , 2010, IEEE Transactions on Automatic Control.

[4]  Edward R. Dougherty,et al.  An Optimization-Based Framework for the Transformation of Incomplete Biological Knowledge into a Probabilistic Structure and Its Application to the Utilization of Gene/Protein Signaling Pathways in Discrete Phenotype Classification , 2015, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[5]  Yeng Chai Soh,et al.  Adaptive Kalman Filtering in Networked Systems With Random Sensor Delays, Multiple Packet Dropouts and Missing Measurements , 2010, IEEE Transactions on Signal Processing.

[6]  B. Anderson,et al.  Optimal Filtering , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[7]  K. Vastola,et al.  Robust Wiener-Kolmogorov theory , 1984, IEEE Trans. Inf. Theory.

[8]  Zeljko M. Durovic,et al.  Robust estimation with unknown noise statistics , 1999, IEEE Trans. Autom. Control..

[9]  Nikolas P. Galatsanos,et al.  A variational approach for Bayesian blind image deconvolution , 2004, IEEE Transactions on Signal Processing.

[10]  Ali H. Sayed,et al.  A framework for state-space estimation with uncertain models , 2001, IEEE Trans. Autom. Control..

[11]  Tong Zhou,et al.  Robust Recursive State Estimation With Random Measurement Droppings , 2014, IEEE Transactions on Automatic Control.

[12]  Edward R. Dougherty,et al.  Intrinsically Optimal Bayesian Robust Filtering , 2014, IEEE Transactions on Signal Processing.

[13]  Raman K. Mehra,et al.  Approaches to adaptive filtering , 1970 .

[14]  Edward R. Dougherty,et al.  Optimal Experimental Design for Gene Regulatory Networks in the Presence of Uncertainty , 2015, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[15]  A. P. Roberts,et al.  An exact equivalence between the discrete- and continuous-time formulations of the Kalman filter , 1978 .

[16]  Edward R. Dougherty,et al.  Inference of Noisy Nonlinear Differential Equation Models for Gene Regulatory Networks Using Genetic Programming and Kalman Filtering , 2008, IEEE Transactions on Signal Processing.

[17]  Tong Zhou,et al.  Structure identification for gene regulatory networks via linearization and robust state estimation , 2014, Autom..

[18]  Edward R. Dougherty,et al.  Bayesian Regression With Network Prior: Optimal Bayesian Filtering Perspective , 2016, IEEE Transactions on Signal Processing.

[19]  T. Kailath,et al.  An innovations approach to least-squares estimation--Part III: Nonlinear estimation in white Gaussian noise , 1971 .

[20]  Joel M. Morris,et al.  The Kalman filter: A robust estimator for some classes of linear quadratic problems , 1976, IEEE Trans. Inf. Theory.

[21]  Edward R. Dougherty,et al.  Quantifying the Objective Cost of Uncertainty in Complex Dynamical Systems , 2013, IEEE Transactions on Signal Processing.

[22]  B. Tapley,et al.  Adaptive sequential estimation with unknown noise statistics , 1976 .

[23]  Thomas Kailath,et al.  A view of three decades of linear filtering theory , 1974, IEEE Trans. Inf. Theory.

[24]  Nan Xiao,et al.  Optimal Filtering for Systems With Multiple Packet Dropouts , 2008, IEEE Transactions on Circuits and Systems II: Express Briefs.

[25]  H. Vincent Poor,et al.  Robust matched filters , 1983, IEEE Trans. Inf. Theory.

[26]  Yuriy S. Shmaliy,et al.  Linear Optimal FIR Estimation of Discrete Time-Invariant State-Space Models , 2010, IEEE Transactions on Signal Processing.

[27]  Roy M. Howard,et al.  Linear System Theory , 1992 .

[28]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[29]  Yuriy S. Shmaliy,et al.  An Iterative Kalman-Like Algorithm Ignoring Noise and Initial Conditions , 2011, IEEE Transactions on Signal Processing.

[30]  H. Vincent Poor,et al.  On minimax robustness: A general approach and applications , 1984, IEEE Trans. Inf. Theory.

[31]  S. Verdú,et al.  Minimax linear observers and regulators for stochastic systems with uncertain second-order statistics , 1984 .

[32]  Saleem A. Kassam,et al.  Robust Wiener filters , 1977 .

[33]  Edward R. Dougherty,et al.  Incorporation of Biological Pathway Knowledge in the Construction of Priors for Optimal Bayesian Classification , 2014, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[34]  T. Kailath,et al.  An innovations approach to least-squares estimation--Part II: Linear smoothing in additive white noise , 1968 .

[35]  Wook Hyun Kwon,et al.  FIR filters and recursive forms for discrete-time state-space models , 1987, Autom..

[36]  Edward R. Dougherty,et al.  Design and Analysis of Robust Binary Filters in the Context of a Prior Distribution for the States of Nature , 2004, Journal of Mathematical Imaging and Vision.

[37]  Edward R. Dougherty,et al.  Erratum to: Efficient experimental design for uncertainty reduction in gene regulatory networks , 2015, BMC Bioinformatics.

[38]  Simo Särkkä,et al.  Recursive Noise Adaptive Kalman Filtering by Variational Bayesian Approximations , 2009, IEEE Transactions on Automatic Control.

[39]  Byung-Jun Yoon,et al.  Efficient experimental design for uncertainty reduction in gene regulatory networks , 2015, BMC Bioinformatics.

[40]  Scott Shald,et al.  The continuous kalman filter as the limit of the discrete kalman filter , 1999 .

[41]  Edward R. Dougherty,et al.  Optimal Objective-Based Experimental Design for Uncertain Dynamical Gene Networks with Experimental Error , 2018, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[42]  H. Akaike,et al.  Comment on "An innovations approach to least-squares estimation, part I: Linear filtering in additive white noise" , 1970 .

[43]  A. H. Mohamed,et al.  Adaptive Kalman Filtering for INS/GPS , 1999 .

[44]  R. E. Kalman,et al.  New Results in Linear Filtering and Prediction Theory , 1961 .

[45]  H. Poor,et al.  Minimax state estimation for linear stochastic systems with noise uncertainty , 1980, 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[46]  Edward R. Dougherty,et al.  Robust optimal granulometric bandpass filters , 2001, Signal Process..

[47]  Edward R. Dougherty,et al.  Bayesian robust optimal linear filters , 2001, Signal Process..

[48]  R. Bellman,et al.  The Riccati Equation , 1986 .

[49]  Robert Grover Brown,et al.  Introduction to random signal analysis and Kalman filtering , 1983 .

[50]  Jing Ma,et al.  Optimal Linear Estimators for Systems With Random Sensor Delays, Multiple Packet Dropouts and Uncertain Observations , 2011, IEEE Transactions on Signal Processing.

[51]  Richard M. Murray,et al.  On a stochastic sensor selection algorithm with applications in sensor scheduling and sensor coverage , 2006, Autom..

[52]  Edward R. Dougherty,et al.  Optimal classifiers with minimum expected error within a Bayesian framework - Part I: Discrete and Gaussian models , 2013, Pattern Recognit..

[53]  A.H. Haddad,et al.  Applied optimal estimation , 1976, Proceedings of the IEEE.

[54]  Wook Hyun Kwon,et al.  Optimal FIR filters for time-varying state-space models , 1990 .

[55]  Edward R. Dougherty,et al.  Optimal experimental design in canonical expansions with applications to signal compression , 2016, 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[56]  R. Mehra On the identification of variances and adaptive Kalman filtering , 1970 .

[57]  H. Poor On robust wiener filtering , 1980 .