论文信息 - Shannon Entropy, Renyi Entropy, and Information

Shannon Entropy, Renyi Entropy, and Information

This memo contains proofs that the Shannon entropy is the limiting case of both the Renyi entropy and the Tsallis entropy, or information. These results are also confirmed experimentally. We conclude with some general observations on the utility of entropy measures. A brief summary of the origins of the concept of physical entropy are provided in an appendix.

[1] Mill Johannes G.A. Van,et al. Transmission Of Information , 1961 .

[2] C. Tsallis. Possible generalization of Boltzmann-Gibbs statistics , 1988 .

[3] H. Nyquist,et al. Certain Topics in Telegraph Transmission Theory , 1928, Transactions of the American Institute of Electrical Engineers.

[4] Guang-Zhong Yang,et al. Computing Covariances for Mutual Information Coregistration , 2004 .

[5] M. Kendall,et al. Kendall's Advanced Theory of Statistics: Volume 1 Distribution Theory , 1987 .

[6] Adrian Cho,et al. A Fresh Take on Disorder, Or Disorderly Science? , 2002, Science.

[7] A. W. Kemp,et al. Kendall's Advanced Theory of Statistics. , 1994 .

[8] Paul A. Viola. Alignment by maximisation of mutual information , 1993 .

[9] Jan Havrda,et al. Quantification method of classification processes. Concept of structural a-entropy , 1967, Kybernetika.

[10] Neil A. Thacker,et al. Empirical Evaluation of Covariance Estimates for Mutual Information Coregistration , 2004, MICCAI.

[11] Sang Joon Kim,et al. A Mathematical Theory of Communication , 2006 .

[12] H. Nyquist,et al. Certain factors affecting telegraph speed , 1924, Journal of the A.I.E.E..