Average Entropy: A New Uncertainty Measure with Application to Image Segmentation

Various modifications have been suggested in the past to extend Shannon entropy to continuous random variables. This article investigates these modifications, and suggests a new entropy measure with the name of average entropy (AE). AE is more general than Shannon entropy in the sense that its definition encompasses both continuous as well as discrete domains. It is additive, positive and attains zero only when the distribution is uniform. The main characteristic of the suggested measure lies in its consistency behavior. Many properties of AE, including its relationship with Kullback–Leibler information measure, are studied. Precise theorems about the vanishing of the conditional AE for both continuous and discrete distributions are provided. Toward the end, the measure is tested for its effectiveness in image segmentation. [Received March 2014. Revised June 2015.]

[1]  Claude E. Shannon,et al.  A Mathematical Theory of Communications , 1948 .

[2]  Edwin T. Jaynes.  Prior Probabilities , 2010, Encyclopedia of Machine Learning.

[3]  Yunmei Chen,et al.  Cumulative residual entropy: a new measure of information , 2004, IEEE Transactions on Information Theory.

[4]  Amiel Feinstein,et al.  Information and information stability of random variables and processes , 1964 .

[5]  Ali El-Zaart,et al.  Thresholding of Medical Images Using Minimum Cross Entropy , 2007 .

[6]  Clinical experience of the first digital mammographic unit in Australia in its first year of use , 2007, The Medical journal of Australia.

[7]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[8]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[9]  G. Crooks.  On Measures of Entropy and Information , 2015 .

[10]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[11]  Bülent Sankur,et al.  Survey over image thresholding techniques and quantitative performance evaluation , 2004, J. Electronic Imaging.

[12]  Terry Caelli,et al.  On the Representation of Image Structures via Scale Space Entropy Conditions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[14]  David Salomon,et al.  A Concise Introduction to Data Compression , 2007, Undergraduate Topics in Computer Science.

[15]  M. Cevdet Ince,et al.  A COMPARATIVE STUDY ON ENTROPIC THRESHOLDING METHODS , 2006 .