A Connection between Feedback Capacity and Kalman Filter for Colored Gaussian Noises

In this paper, we establish a connection between the feedback capacity of additive colored Gaussian noise channels and the Kalman filters with additive colored Gaussian noises. In light of this, we are able to provide lower bounds on feedback capacity of such channels with finite-order auto-regressive moving average colored noises, and the bounds are seen to be consistent with various existing results in the literature; particularly, the bound is tight in the case of first-order auto-regressive moving average colored noises. On the other hand, the Kalman filtering systems, after certain equivalence transformations, can be employed as recursive coding schemes/algorithms to achieve the lower bounds. In general, our results provide an alternative perspective while pointing to potentially tighter bounds for the feedback capacity problem.

[1]  Thomas M. Cover,et al.  Gaussian feedback capacity , 1989, IEEE Trans. Inf. Theory.

[2]  S. Butman.  Linear feedback rate bounds for regressive channels , 1976 .

[3]  Christos K. Kourtellaris,et al.  Sequential Necessary and Sufficient Conditions for Capacity Achieving Distributions of Channels With Memory and Feedback , 2016, IEEE Transactions on Information Theory.

[4]  Christos K. Kourtellaris,et al.  Information Structures of Capacity Achieving Distributions for Feedback Channels with Memory and Transmission Cost: Stochastic Optimal Control & Variational Equalities-Part I , 2015, ArXiv.

[5]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[6]  Young-Han Kim,et al.  Feedback Capacity of Stationary Gaussian Channels , 2006, 2006 IEEE International Symposium on Information Theory.

[7]  Petros G. Voulgaris,et al.  On optimal ℓ∞ to ℓ∞ filtering , 1995, Autom..

[8]  Jie Chen,et al.  An Integral Characterization of Optimal Error Covariance by Kalman Filtering , 2018, 2018 Annual American Control Conference (ACC).

[9]  Ather Gattami.  Feedback Capacity of Gaussian Channels Revisited , 2019, IEEE Transactions on Information Theory.

[10]  Nicola Elia,et al.  Youla Coding and Computation of Gaussian Feedback Capacity , 2018, IEEE Transactions on Information Theory.

[11]  Sekhar Tatikonda,et al.  The Capacity of Channels With Feedback , 2006, IEEE Transactions on Information Theory.

[12]  Takashi Tanaka,et al.  Some Results on the Computation of Feedback Capacity of Gaussian Channels with Memory , 2018, 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[13]  Tao Liu,et al.  Feedback Capacity of Stationary Gaussian Channels Further Examined , 2019, IEEE Transactions on Information Theory.

[14]  Nicola Elia,et al.  Convergence of Fundamental Limitations in Feedback Communication, Estimation, and Feedback Control over Gaussian Channels , 2009, Commun. Inf. Syst..

[15]  Sekhar Tatikonda,et al.  On the Feedback Capacity of Power-Constrained Gaussian Noise Channels With Memory , 2007, IEEE Transactions on Information Theory.

[16]  Nicola Elia,et al.  When bode meets shannon: control-oriented feedback communication schemes , 2004, IEEE Transactions on Automatic Control.

[17]  Richard M. Murray,et al.  Feedback Systems An Introduction for Scientists and Engineers , 2007 .

[18]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[19]  Young Han Kim,et al.  Feedback capacity of the first-order moving average Gaussian channel , 2004, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[20]  Sekhar Tatikonda,et al.  Capacity-Achieving Feedback Schemes for Gaussian Finite-State Markov Channels With Channel State Information , 2008, IEEE Transactions on Information Theory.

[21]  P. P. Vaidyanathan,et al.  The Theory of Linear Prediction , 2008, Synthesis Lectures on Signal Processing.

[22]  Massimo Franceschetti,et al.  Control-Theoretic Approach to Communication With Feedback , 2012, IEEE Transactions on Automatic Control.

[23]  Jie Chen,et al.  Towards Integrating Control and Information Theories , 2017 .

[24]  Stanley A. Butman,et al.  A general formulation of linear feedback communication systems with solutions , 1969, IEEE Trans. Inf. Theory.

[25]  Quanyan Zhu,et al.  Generic Variance Bounds on Estimation and Prediction Errors in Time Series Analysis: An Entropy Perspective , 2019, 2019 IEEE Information Theory Workshop (ITW).