Basic inequalities for weighted entropies

The concept of weighted entropy takes into account values of different outcomes, i.e., makes entropy context-dependent, through the weight function. In this paper, we establish a number of simple inequalities for the weighted entropies (general as well as specific), mirroring similar bounds on standard (Shannon) entropies and related quantities. The required assumptions are written in terms of various expectations of weight functions. Examples are weighted Ky Fan and weighted Hadamard inequalities involving determinants of positive-definite matrices, and weighted Cramér-Rao inequalities involving the weighted Fisher information matrix.

[1]  Silviu Guiasu,et al.  A quantitative-qualitative measure of information in cybernetic systems (Corresp.) , 1968, IEEE Trans. Inf. Theory.

[2]  R. P. Singh,et al.  On parametric weighted information improvement , 1992, Inf. Sci..

[3]  C. Geiss,et al.  An introduction to probability theory , 2008 .

[4]  Jagdish Mitter,et al.  On Measures of "Useful" Information , 1978, Inf. Control..

[5]  Yuri M. Suhov,et al.  Entropy-power inequality for weighted entropy , 2015, ArXiv.

[6]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[7]  Paul G. Hoel,et al.  Introduction to Probability Theory , 1972 .

[8]  H. C. Taneja,et al.  Characterization of the quantitative-qualitative measure of inaccuracy for discrete generalized probability distributions , 1986 .

[9]  T. Cover,et al.  IEEE TRANSACTIONSON INFORMATIONTHEORY,VOL. IT-30,N0. 6,NOVEmER1984 Correspondence On the Similarity of the Entropy Power Inequality The preceeding equations allow the entropy power inequality and the Brunn-Minkowski Inequality to be rewritten in the equiv , 2022 .

[10]  Yuri M. Suhov,et al.  Weighted Gaussian entropy and determinant inequalities , 2015, Aequationes mathematicae.

[11]  T. Cover,et al.  Determinant inequalities via information theory , 1988 .

[12]  K. Fan,et al.  Maximum Properties and Inequalities for the Eigenvalues of Completely Continuous Operators. , 1951, Proceedings of the National Academy of Sciences of the United States of America.

[13]  Ram Zamir,et al.  A Proof of the Fisher Information Inequality via a Data Processing Argument , 1998, IEEE Trans. Inf. Theory.

[14]  K. Fan On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations: II. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Prasanna K. Sahoo,et al.  On the general solution of a functional equation connected to sum form information measures on open domain—III , 1986 .

[16]  M. Moslehian Ky Fan inequalities , 2011, 1108.1467.

[17]  Jagat Narain Kapur,et al.  Measures of information and their applications , 1994 .

[18]  K. Fan On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations I. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[19]  Adriana Clim,et al.  Weighted entropy with application , 2008 .

[20]  G Frizelle,et al.  The measurement of complexity in production and other commercial systems , 2008, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[21]  Aimé Fuchs,et al.  L'inégalité de Kullback. Application à la théorie de l'estimation , 1970 .

[22]  Don H. Johnson,et al.  When does interval coding occur? , 2003, Neurocomputing.

[23]  I. Stuhl,et al.  Weight functions and log-optimal investment portfolios , 2015 .

[24]  Amir Dembo,et al.  Information theoretic inequalities , 1991, IEEE Trans. Inf. Theory.

[25]  Mark Kelbert,et al.  Information Theory and Coding by Example , 2013 .

[26]  Maria Longobardi,et al.  Entropy-based measure of uncertainty in past lifetime distributions , 2002, Journal of Applied Probability.

[27]  Mark Kelbert,et al.  Continuity of Mutual Entropy in the Limiting Signal-To-Noise Ratio Regimes , 2011 .

[28]  Krikamol Muandet,et al.  Query Selection via Weighted Entropy in Graph-Based Semi-supervised Classification , 2009, ACML.

[29]  Y. Suhov,et al.  An entropic measurement of queueing behaviour in a class of manufacturing operations , 2001, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.