Arbitrarily Tight Bounds on Differential Entropy of Gaussian Mixtures

A sequence of lower and upper bounds is derived on the differential entropy of a Gaussian mixture where the Gaussian components only differ in mean values. As the sequence index increases, the computational complexity of the bounds increases; however, the gap between the lower and upper bounds becomes vanishingly small. We address the applications of these bounds in several communication scenarios where the transmitters utilize Pulse Amplitude Modulation (PAM) constellations to transmit data.

[1]  Shlomo Shamai,et al.  Transition points in the capacity-achieving distribution for the peak-power limited AWGN and free-space optical intensity channels , 2010, Probl. Inf. Transm..

[2]  R. M. Dudley,et al.  Real Analysis and Probability , 1989 .

[3]  Hugh F. Durrant-Whyte,et al.  On entropy approximation for Gaussian mixture random vectors , 2008, 2008 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems.

[4]  Dudley,et al.  Real Analysis and Probability: Measurability: Borel Isomorphism and Analytic Sets , 2002 .

[5]  Shlomo Shamai,et al.  Mutual information and minimum mean-square error in Gaussian channels , 2004, IEEE Transactions on Information Theory.

[6]  James F. Epperson,et al.  An Introduction to Numerical Methods and Analysis , 2001 .

[7]  Joel G. Smith,et al.  The Information Capacity of Amplitude- and Variance-Constrained Scalar Gaussian Channels , 1971, Inf. Control..

[8]  Jonathan M. Nichols,et al.  Calculation of Differential Entropy for a Mixed Gaussian Distribution , 2008, Entropy.

[9]  Thomas M. Cover,et al.  Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing) , 2006 .

[10]  Kamyar Moshksar,et al.  Capacity-Achieving Distributions in Gaussian Multiple Access Channel With Peak Power Constraints , 2014, IEEE Transactions on Information Theory.

[11]  S. Verdú,et al.  The impact of constellation cardinality on Gaussian channel capacity , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[12]  Milton Abramowitz,et al.  Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables , 1964 .

[13]  N. Temme.  Special Functions: An Introduction to the Classical Functions of Mathematical Physics , 1996 .

[14]  E. Saff,et al.  Fundamentals of complex analysis with applications to engineering and science , 2003 .

[15]  Kamyar Moshksar,et al.  On The effect of self-interference in Gaussian two-way channels with erased outputs , 2013, 2013 IEEE International Symposium on Information Theory.

[16]  Sanjeev R. Kulkarni,et al.  Universal Estimation of Information Measures for Analog Sources , 2009, Found. Trends Commun. Inf. Theory.

[17]  M. Raginsky,et al.  On the information capacity of Gaussian channels under small peak power constraints , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[18]  Giuseppe Thadeu Freitas de Abreu,et al.  Jensen-cotes upper and lower bounds on the gaussian Q-function and related functions , 2009, IEEE Transactions on Communications.