Weighted entropy: basic inequalities

This paper represents an extended version of an earlier note [10]. The concept of weighted entropy takes into account values of different outcomes, i.e., makes entropy context-dependent, through the weight function. We analyse analogs of the Fisher information inequality and entropy power inequality for the weighted entropy and discuss connections with weighted Lieb’s splitting inequality. The concepts of rates of the weighted entropy and information are also discussed.

[1]  E. Lieb.  Proof of an entropy conjecture of Wehrl , 1978 .

[2]  David N. Nawrocki,et al.  State-value weighted entropy as a measure of investment risk , 1986 .

[3]  Amir Dembo,et al.  Simple proof of the concavity of the entropy power with respect to Gaussian noise , 1989, IEEE Trans. Inf. Theory.

[4]  Amir Dembo,et al.  Information theoretic inequalities , 1991, IEEE Trans. Inf. Theory.

[5]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[6]  Ram Zamir,et al.  A Proof of the Fisher Information Inequality via a Data Processing Argument , 1998, IEEE Trans. Inf. Theory.

[7]  Cédric Villani,et al.  A short proof of the "Concavity of entropy power" , 2000, IEEE Trans. Inf. Theory.

[8]  Y. Suhov,et al.  An entropic measurement of queueing behaviour in a class of manufacturing operations , 2001, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[9]  Lei Yang,et al.  Weighted Information Entropy: A Method for Estimating the Complex Degree of Infrared Images' Backgrounds , 2005, ICIAR.

[10]  Sergio Verdú,et al.  A simple proof of the entropy-power inequality , 2006, IEEE Transactions on Information Theory.

[11]  G Frizelle,et al.  The measurement of complexity in production and other commercial systems , 2008, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[12]  Olivier Rioul,et al.  Information Theoretic Proofs of Entropy Power Inequalities , 2007, IEEE Transactions on Information Theory.

[13]  Mark Kelbert,et al.  Continuity of Mutual Entropy in the Limiting Signal-To-Noise Ratio Regimes , 2011 .

[14]  Geong Sen Poh,et al.  Weighted Entropy-based Measure for Image Segmentation , 2012 .

[15]  Mark Kelbert,et al.  Information Theory and Coding by Example , 2013 .

[16]  Constantinos Patsakis,et al.  The Role of Weighted Entropy in Security Quantification , 2013 .

[17]  Jesmin F. Khan,et al.  Weighted entropy for segmentation evaluation , 2014 .

[18]  Keith R Shockley,et al.  Using Weighted Entropy to Rank Chemicals in Quantitative High-Throughput Screening Experiments , 2014, Journal of biomolecular screening.

[19]  Yuri M. Suhov,et al.  Basic inequalities for weighted entropies , 2015, ArXiv.

[20]  Yuri M. Suhov,et al.  An extension of the Ky Fan inequality , 2015, ArXiv.

[21]  Yuri M. Suhov,et al.  Entropy-power inequality for weighted entropy , 2015, ArXiv.

[22]  Po-Hsiang Tsui,et al.  Ultrasound Detection of Scatterer Concentration by Weighted Entropy , 2015, Entropy.

[23]  Yuri M. Suhov,et al.  Weighted information and entropy rates , 2016, ArXiv.

[24]  Yuri M. Suhov,et al.  Weighted Entropy and its Use in Computer Science and Beyond , 2017, ACMPT.

[25]  Mark Kelbert,et al.  Weighted entropy and optimal portfolios for risk-averse Kelly investments , 2017, 1708.03813.

[26]  Yuri M. Suhov,et al.  Weighted Gaussian entropy and determinant inequalities , 2015, Aequationes mathematicae.