On the entropy of sums

It is shown that the entropy of a sum of independent random vectors is a submodular set function, and upper bounds on the entropy of sums are obtained as a result in both discrete and continuous settings. These inequalities complement the lower bounds provided by the entropy power inequalities of Madiman and Barron (2007). As applications, new inequalities for the determinants of sums of positive-definite matrices are presented.

[1]  Richard Bellman,et al.  Introduction to Matrix Analysis , 1972 .

[2]  Mokshay M. Madiman,et al.  Information Inequalities for Joint Distributions, With Interpretations and Applications , 2008, IEEE Transactions on Information Theory.

[3]  Fan Chung Graham,et al.  Some intersection theorems for ordered sets and graphs , 1986, J. Comb. Theory, Ser. A.

[4]  Satoru Fujishige,et al.  Polymatroidal Dependence Structure of a Set of Random Variables , 1978, Inf. Control..

[5]  Randall Dougherty,et al.  Networks, Matroids, and Non-Shannon Information Inequalities , 2007, IEEE Transactions on Information Theory.

[6]  Béla Bollobás,et al.  Compressions and isoperimetric inequalities , 1990, J. Comb. Theory, Ser. A.

[7]  Te Sun Han.  Nonnegative Entropy Measures of Multivariate Symmetric Correlations , 1978, Inf. Control..

[8]  Amir Dembo,et al.  Information theoretic inequalities , 1991, IEEE Trans. Inf. Theory.

[9]  Abbas El Gamal,et al.  An information - theoretic proof of Hadamard's inequality , 1983, IEEE Trans. Inf. Theory.

[10]  Mokshay M. Madiman,et al.  Generalized Entropy Power Inequalities and Monotonicity Properties of Information , 2006, IEEE Transactions on Information Theory.

[11]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[12]  Zhen Zhang,et al.  On Characterization of Entropy Function via Information Inequalities , 1998, IEEE Trans. Inf. Theory.

[13]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[14]  A. J. Stam.  Some Inequalities Satisfied by the Quantities of Information of Fisher and Shannon , 1959, Inf. Control..

[15]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[16]  F. Matús,et al.  Two Constructions on Limits of Entropy Functions , 2007, IEEE Transactions on Information Theory.

[17]  K. Ball,et al.  Solution of Shannon's problem on the monotonicity of entropy , 2004 .

[18]  Olivier Rioul.  A Simple Proof of the Entropy-Power Inequality via Properties of Mutual Information , 2007, 2007 IEEE International Symposium on Information Theory.