Control-theoretic Approach to Communication with Feedback: Fundamental Limits and Code Design

Feedback communication is studied from a control-theoretic perspective, mapping the communication problem to a control problem in which the control signal is received through the same noisy channel as in the communication problem, and the (nonlinear and time-varying) dynamics of the system determine a subclass of encoders available at the transmitter. The MMSE capacity is defined to be the supremum exponential decay rate of the mean square decoding error. This is upper bounded by the information-theoretic feedback capacity, which is the supremum of the achievable rates. A sufficient condition is provided under which the upper bound holds with equality. For the special class of stationary Gaussian channels, a simple application of Bode's integral formula shows that the feedback capacity, recently characterized by Kim, is equal to the maximum instability that can be tolerated by the controller under a given power constraint. Finally, the control mapping is generalized to the N-sender AWGN multiple access channel. It is shown that Kramer's code for this channel, which is known to be sum rate optimal in the class of generalized linear feedback codes, can be obtained by solving a linear quadratic Gaussian control problem.

[1]  Shlomo Shamai,et al.  Mutual information and minimum mean-square error in Gaussian channels , 2004, IEEE Transactions on Information Theory.

[2]  Wei Wu,et al.  Gaussian interference networks with feedback: Duality, sum capacity and dynamic team problems , 2005 .

[3]  P. Iglesias.  An analogue of Bode's integral for stable nonlinear systems: relations to entropy , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[4]  T. Duncan.  ON THE CALCULATION OF MUTUAL INFORMATION , 1970 .

[5]  Sanjoy K. Mitter,et al.  Control with Limited Information , 2001, Eur. J. Control.

[6]  Young-Han Kim.  Feedback Capacity of Stationary Gaussian Channels , 2006, ISIT.

[7]  Thomas Kailath,et al.  A coding scheme for additive noise channels with feedback-I: No bandwidth constraint , 1966, IEEE Trans. Inf. Theory.

[8]  Sekhar Tatikonda,et al.  The Capacity of Channels With Feedback , 2006, IEEE Transactions on Information Theory.

[9]  Abbas El Gamal,et al.  Lecture Notes on Network Information Theory , 2010, ArXiv.

[10]  Munther A. Dahleh,et al.  Fundamental Limitations of Disturbance Attenuation in the Presence of Side Information , 2005, CDC 2005.

[11]  H. Witsenhausen.  A Counterexample in Stochastic Optimum Control , 1968 .

[12]  Bruce A. Francis,et al.  Feedback Control Theory , 1992 .

[13]  Thomas M. Cover,et al.  Gaussian feedback capacity , 1989, IEEE Trans. Inf. Theory.

[14]  H. W. Bode,et al.  Network analysis and feedback amplifier design , 1945 .

[15]  Munther A. Dahleh,et al.  Feedback Control in the Presence of Noisy Channels: “Bode-Like” Fundamental Limitations of Performance , 2008, IEEE Transactions on Automatic Control.

[16]  H. Witsenhausen.  Separation of estimation and control for discrete time systems , 1971 .

[17]  J. Freudenberg,et al.  Frequency Domain Properties of Scalar and Multivariable Feedback Systems , 1988 .

[18]  Gerhard Kramer,et al.  Feedback strategies for white Gaussian interference networks , 2002, IEEE Trans. Inf. Theory.

[19]  G. David Forney,et al.  On the role of MMSE estimation in approaching the information-theoretic limits of linear Gaussian channels: Shannon meets Wiener , 2004, ArXiv.

[20]  Shinji Hara,et al.  Properties of sensitivity and complementary sensitivity functions in single-input single-output digital control systems , 1988 .

[21]  Nicola Elia,et al.  When bode meets shannon: control-oriented feedback communication schemes , 2004, IEEE Transactions on Automatic Control.

[22]  Nigel J. Newton,et al.  Information and Entropy Flow in the Kalman–Bucy Filter , 2005 .

[23]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[24]  Todd P. Coleman,et al.  A stochastic control viewpoint on ‘Posterior Matching’-style feedback communication schemes , 2009, 2009 IEEE International Symposium on Information Theory.

[25]  Sanjoy K. Mitter,et al.  A Variational Approach to Nonlinear Estimation , 2003, SIAM J. Control. Optim..

[26]  Lawrence H. Ozarow,et al.  The capacity of the white Gaussian multiple access channel with feedback , 1984, IEEE Trans. Inf. Theory.

[27]  Meir Feder,et al.  Optimal Feedback Communication Via Posterior Matching , 2009, IEEE Transactions on Information Theory.

[28]  J. Pieter M. Schalkwijk,et al.  A coding scheme for additive noise channels with feedback-II: Band-limited signals , 1966, IEEE Trans. Inf. Theory.

[29]  Munther A. Dahleh,et al.  Fundamental Limitations of Disturbance Attenuation in the Presence of Side Information , 2005, IEEE Transactions on Automatic Control.

[30]  Moshe Zakai,et al.  On mutual information, likelihood ratios, and estimation error for the additive Gaussian channel , 2004, IEEE Transactions on Information Theory.

[31]  Young-Han Kim,et al.  Feedback Capacity of Stationary Gaussian Channels , 2006, 2006 IEEE International Symposium on Information Theory.

[32]  Pablo A. Iglesias,et al.  Nonlinear extension of Bode's integral based on an information-theoretic interpretation , 2003, Syst. Control. Lett..

[33]  Tara Javidi,et al.  Linear sum capacity for Gaussian multiple access channel with feedback , 2010, 2010 IEEE International Symposium on Information Theory.

[34]  Prashant G. Mehta,et al.  Bode-Like Fundamental Performance Limitations in Control of Nonlinear Systems , 2010, IEEE Transactions on Automatic Control.