Gradient of mutual information in linear vector Gaussian channels

This paper considers a general linear vector Gaussian channel with arbitrary signaling and pursues two closely related goals: i) closed-form expressions for the gradient of the mutual information with respect to arbitrary parameters of the system, and ii) fundamental connections between information theory and estimation theory. Generalizing the fundamental relationship recently unveiled by Guo, Shamai, and Verdu, we show that the gradient of the mutual information with respect to the channel matrix is equal to the product of the channel matrix and the error covariance matrix of the estimate of the input given the output

[1]  A. J. Stam.  Some Inequalities Satisfied by the Quantities of Information of Fisher and Shannon , 1959, Inf. Control..

[2]  W. Rudin.  Principles of mathematical analysis , 1964 .

[3]  Nelson M. Blachman,et al.  The convolution inequality for entropy powers , 1965, IEEE Trans. Inf. Theory.

[4]  Jacob Ziv,et al.  Some lower bounds on signal parameter estimation , 1969, IEEE Trans. Inf. Theory.

[5]  T. Duncan.  ON THE CALCULATION OF MUTUAL INFORMATION , 1970 .

[6]  Jacob Ziv,et al.  Mutual information of the white Gaussian channel with and without feedback , 1971, IEEE Trans. Inf. Theory.

[7]  Kyong-Hwa Lee,et al.  Optimal Linear Coding for Vector Channels , 1976, IEEE Trans. Commun..

[8]  B. A. D. H. Brandwood.  A complex gradient operator and its applica-tion in adaptive array theory , 1983 .

[9]  Max H. M. Costa,et al.  A new entropy power inequality , 1985, IEEE Trans. Inf. Theory.

[10]  J. Salz,et al.  Digital transmission over cross-coupled linear channels , 1985, AT&T Technical Journal.

[11]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[12]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics (Revised Edition) , 1999 .

[13]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[14]  Amir Dembo,et al.  Information theoretic inequalities , 1991, IEEE Trans. Inf. Theory.

[15]  S. Kay.  Fundamentals of statistical signal processing: estimation theory , 1993 .

[16]  H. Vincent Poor,et al.  An introduction to signal detection and estimation (2nd ed.) , 1994 .

[17]  Jian Yang,et al.  On joint transmitter and receiver optimization for multiple-input-multiple-output (MIMO) transmission systems , 1994, IEEE Trans. Commun..

[18]  Sergio Verdú,et al.  Sensitivity of channel capacity , 1995, IEEE Trans. Inf. Theory.

[19]  Anna Scaglione,et al.  Redundant filterbank precoders and equalizers. I. Unification and optimal designs , 1999, IEEE Trans. Signal Process..

[20]  O. Johnson,et al.  Entropy and Random Vectors , 2001 .

[21]  Sergio Verdú,et al.  Spectral efficiency in the wideband regime , 2002, IEEE Trans. Inf. Theory.

[22]  Mikael Skoglund,et al.  Combining beamforming and orthogonal space-time block coding , 2002, IEEE Trans. Inf. Theory.

[23]  John M. Cioffi,et al.  Joint Tx-Rx beamforming design for multicarrier MIMO channels: a unified framework for convex optimization , 2003, IEEE Trans. Signal Process..

[24]  Sergio Verdú,et al.  Second-order asymptotics of mutual information , 2004, IEEE Transactions on Information Theory.

[25]  Daniel Pérez Palomar,et al.  Gradient of mutual information in linear vector Gaussian channels , 2006, IEEE Transactions on Information Theory.

[26]  Shlomo Shamai,et al.  Mutual information and minimum mean-square error in Gaussian channels , 2004, IEEE Transactions on Information Theory.

[27]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.