Entropy and Its Discontents: A Note on Definitions

The routine definitions of Shannon entropy for both discrete and continuous probability laws show inconsistencies that make them not reciprocally coherent. We propose a few possible modifications of these quantities so that: (1) they no longer show incongruities; and (2) they go one into the other in a suitable limit as the result of a renormalization. The properties of the new quantities would slightly differ from that of the usual entropies in a few other respects.

[1]  D. Vernon.  Inform , 1995, Encyclopedia of the UN Sustainable Development Goals.

[2]  Peter Orbanz.  Probability Theory II , 2011 .

[3]  Philippe Jacquet,et al.  Entropy Computations via Analytic Depoissonization , 1999, IEEE Trans. Inf. Theory.

[4]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[5]  Remo Guidieri.  Res , 1995, RES: Anthropology and Aesthetics.

[6]  C. Tsallis.  Possible generalization of Boltzmann-Gibbs statistics , 1988 .

[7]  Thierry Chonavel,et al.  Generalized Cumulative Residual Entropy for Distributions with Unrestricted Supports , 2008, J. Electr. Comput. Eng..

[8]  Vadas Gintautas,et al.  Identification of functional information subgraphs in complex networks. , 2007, Physical review letters.

[9]  Zbigniew Gołębiewski,et al.  On Bernoulli Sums and Bernstein Polynomials , 2012 .