Thermodynamics-Based Evaluation of Various Improved Shannon Entropies for Configurational Information of Gray-Level Images

The quality of an image affects its utility and image quality assessment has been a hot research topic for many years. One widely used measure for image quality assessment is Shannon entropy, which has a well-established information-theoretic basis. The value of this entropy can be interpreted as the amount of information. However, Shannon entropy is badly adapted to information measurement in images, because it captures only the compositional information of an image and ignores the configurational aspect. To fix this problem, improved Shannon entropies have been actively proposed in the last few decades, but a thorough evaluation of their performance is still lacking. This study presents such an evaluation, involving twenty-three improved Shannon entropies based on various tools such as gray-level co-occurrence matrices and local binary patterns. For the evaluation, we proposed: (a) a strategy to generate testing (gray-level) images by simulating the mixing of ideal gases in thermodynamics; (b) three criteria consisting of validity, reliability, and ability to capture configurational disorder; and (c) three measures to assess the fulfillment of each criterion. The evaluation results show only the improved entropies based on local binary patterns are invalid for use in quantifying the configurational information of images, and the best variant of Shannon entropy in terms of reliability and ability is the one based on the average distance between same/different-value pixels. These conclusions are theoretically important in setting a direction for the future research on improving entropy and are practically useful in selecting an effective entropy for various image processing applications.

[1]  Bin Jiang,et al.  How Complex Is a Fractal? Head/tail Breaks and Fractional Hierarchy , 2017, Journal of Geovisualization and Spatial Analysis.

[2]  Samuel A. Cushman.  Thermodynamics in landscape ecology: the importance of integrating measurement and modeling of landscape entropy , 2014, Landscape Ecology.

[3]  David Brosset,et al.  International Journal of Geographical Information Science Local and Global Spatio-temporal Entropy Indices Based on Distance- Ratios and Co-occurrences Distributions Local and Global Spatio-temporal Entropy Indices Based on Distance-ratios and Co-occurrences Distributions , 2022 .

[4]  M. Quweider.  Spatial Entropy-based Cost Function for Gray and Color Image Segmentation with Dynamic Optimal Partitioning , 2012 .

[5]  Zhilin Li,et al.  Fractality and Self-Similarity in the Structure of Road Networks , 2012 .

[6]  Yanguang Chen,et al.  A new model of urban population density indicating latent fractal structure , 2009, 1609.07697.

[7]  Robert I. Damper,et al.  Band Selection for Hyperspectral Image Classification Using Mutual Information , 2006, IEEE Geoscience and Remote Sensing Letters.

[8]  Azriel Rosenfeld,et al.  Pixel Classification Based on Gray Level and Local ``Busyness'' , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Zongjian Lin,et al.  Quantifying Degrees of Information in Remote Sensing Imagery , 2008 .

[10]  A. D. Brink,et al.  Using spatial information as an aid to maximum entropy image threshold selection , 1996, Pattern Recognit. Lett..

[11]  Gang Liu,et al.  Characterizing Traffic Conditions from the Perspective of Spatial-Temporal Heterogeneity , 2016, ISPRS Int. J. Geo Inf..

[12]  Di Wu,et al.  Entropies of the Chinese Land Use/Cover Change from 1990 to 2010 at a County Level , 2017, Entropy.

[13]  Yi Chai,et al.  A Novel Geometric Dictionary Construction Approach for Sparse Representation Based Image Fusion , 2017, Entropy.

[14]  Lars Harrie,et al.  An evaluation of measures for quantifying map information , 2010 .

[15]  Qian Du,et al.  Local Binary Patterns and Extreme Learning Machine for Hyperspectral Imagery Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Yanguang Chen.  The rank-size scaling law and entropy-maximizing principle , 2011, 1104.5630.

[17]  Lei Zhang,et al.  Deep Convolutional Neural Models for Picture-Quality Prediction: Challenges and Solutions to Data-Driven Image Quality Assessment , 2017, IEEE Signal Processing Magazine.

[18]  Chenghu Zhou,et al.  A strategy for raster-based geocomputation under different parallel computing platforms , 2014, Int. J. Geogr. Inf. Sci..

[19]  Peichao Gao,et al.  A hierarchy-based solution to calculate the configurational entropy of landscape gradients , 2017, Landscape Ecology.

[20]  R. Panero,et al.  A quantitative approach to the dynamics of neurite sprouting induced by a neurotrophic factor , 2009, Journal of Neuroscience Methods.

[21]  Yanguang Chen,et al.  Defining urban and rural regions by multifractal spectrums of urbanization , 2015 .

[22]  Jiejing Wang,et al.  Multifractal Characterization of Urban Form and Growth: The Case of Beijing , 2013 .

[23]  Lei Tang,et al.  Accelerating the computation of multi-scale visual curvature for simplifying a large set of polylines with Hadoop , 2015 .

[24]  Yongyi Yang,et al.  Learning a Channelized Observer for Image Quality Assessment , 2009, IEEE Transactions on Medical Imaging.

[25]  Mitsuru Ikeda,et al.  Quantification of image quality using information theory , 2011, Australasian Physical & Engineering Sciences in Medicine.

[26]  Gang Liu,et al.  Scale-free networks of the earth’s surface , 2016 .

[27]  E. C. Pielou.  The use of information theory in the study of the diversity of biological populations , 1967 .

[28]  Peichao Gao,et al.  A Comparative Study of Geographical Information Services in Public and Private Clouds , 2016 .

[29]  Chein-I Chang,et al.  An information-theoretic approach to spectral variability, similarity, and discrimination for hyperspectral image analysis , 2000, IEEE Trans. Inf. Theory.

[30]  Jeffrey G. Masek,et al.  Landsat-7 ETM+ as an observatory for land cover: Initial radiometric and geometric comparisons with Landsat-5 Thematic Mapper , 2001 .

[31]  Xiaogang Wang,et al.  Visual Importance and Distortion Guided Deep Image Quality Assessment Framework , 2017, IEEE Transactions on Multimedia.

[32]  Adolfo Martínez Usó,et al.  Clustering-Based Hyperspectral Band Selection Using Information Measures , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[33]  Christophe Claramunt,et al.  A Spatial Form of Diversity , 2005, COSIT.

[34]  Na Yao,et al.  RESEARCH ON DIFFERENTIAL CODING METHOD FOR SATELLITE REMOTE SENSING DATA COMPRESSION , 2012 .

[35]  Michael F. Goodchild,et al.  The Nature and Value of Geographic Information , 2003, Foundations of Geographic Information Science.

[36]  K. Bailey.  ENTROPY SYSTEMS THEORY , 2022 .

[37]  Jingfeng Huang,et al.  A satellite-based geographically weighted regression model for regional PM2.5 estimation over the Pearl River Delta region in China , 2014 .

[38]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[39]  Qiming Zhou,et al.  Delineating Urban Fringe Area by Land Cover Information Entropy - An Empirical Study of Guangzhou-Foshan Metropolitan Area, China , 2016, ISPRS Int. J. Geo Inf..

[40]  A. D. Brink,et al.  Minimum spatial entropy threshold selection , 1995 .

[41]  Wei Zhang,et al.  Robust Feature Matching Method for SAR and Optical Images by Using Gaussian-Gamma-Shaped Bi-Windows-Based Descriptor and Geometric Constraint , 2017, Remote. Sens..

[42]  Jesús Álvarez-Mozos,et al.  Automatic Detection of Uprooted Orchards Based on Orthophoto Texture Analysis , 2017, Remote. Sens..

[43]  J. C. Price,et al.  Comparison of the Information Content of Data from the LANDSAT-4 Thematic Mapper And the Multispectral Scanner , 1984, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Zhilin Li,et al.  Quantitative measures for spatial information of maps , 2002, Int. J. Geogr. Inf. Sci..

[45]  Jiping Liu,et al.  Method for Measuring the Information Content of Terrain from Digital Elevation Models , 2015, Entropy.

[46]  S. Cushman.  Calculating the configurational entropy of a landscape mosaic , 2016, Landscape Ecology.

[47]  Yanguang Chen,et al.  Analogies between urban hierarchies and river networks: Fractals, symmetry, and self-organized criticality , 2009 .

[48]  Terrell L. Hill.  Lectures on Matter and Equilibrium , 1966 .

[49]  Mert R. Sabuncu,et al.  Entropy-based Image Registration , 2006 .

[50]  Michael O. Finkelstein.  Basic Concepts of Probability and Statistics in the Law , 2009 .

[51]  Liu Qiliang,et al.  Entropy-based Cartographic Communication Models: Evolution from Special to General Cartographic Information Theory , 2016 .

[52]  Mateu Sbert,et al.  Information Theory Tools for Image Processing , 2014, Information Theory Tools for Image Processing.

[53]  S. Noorizadeh,et al.  Shannon entropy as a new measure of aromaticity, Shannon aromaticity. , 2010, Physical chemistry chemical physics : PCCP.

[54]  Zhou Wang,et al.  Why is image quality assessment so difficult? , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[55]  Hongrui Zhao,et al.  Unified Metrics for Characterizing the Fractal Nature of Geographic Features , 2017 .

[56]  B. Mandelbrot.  How Long Is the Coast of Britain? Statistical Self-Similarity and Fractional Dimension , 1967, Science.

[57]  J. G. Lyon,et al.  Hyperspectral Remote Sensing of Vegetation , 2011 .

[58]  Ahmed S. Abutableb.  Automatic thresholding of gray-level pictures using two-dimensional entropy , 1989 .

[59]  Irwin Epstein,et al.  Research Techniques for Clinical Social Workers , 1980 .

[60]  Niu Fangqu.  Verification of Raster-based Map Information Measurement , 2006 .

[61]  Jian Feng,et al.  Understanding the Fractal Dimensions of Urban Forms through Spatial Entropy , 2017, Entropy.

[62]  Yanguang Chen,et al.  Multi-Scaling Allometric Analysis for Urban and Regional Development: Model, Method, and Applications , 2011, 1105.5445.

[63]  Qiong Cao,et al.  Optimal Decision Fusion for Urban Land-Use/Land-Cover Classification Based on Adaptive Differential Evolution Using Hyperspectral and LiDAR Data , 2017, Remote. Sens..

[64]  Xin Wang,et al.  Spatial Entropy-Based Clustering for Mining Data with Spatial Correlation , 2011, PAKDD.

[65]  Duk-jin Kim,et al.  Detection of Durable and Permanent Changes in Urban Areas Using Multitemporal Polarimetric UAVSAR Data , 2016, IEEE Geoscience and Remote Sensing Letters.

[66]  Harvey Gould,et al.  Statistical and Thermal Physics , 2010 .

[67]  Waldo R. Tobler.  Introductory Comments on Information Theory and Cartography , 1997 .

[68]  Peichao Gao,et al.  CRG Index: A More Sensitive Ht-Index for Enabling Dynamic Views of Geographic Features , 2016 .

[69]  Yanguang Chen,et al.  POWER-LAW DISTRIBUTIONS BASED ON EXPONENTIAL DISTRIBUTIONS: LATENT SCALING, SPURIOUS ZIPF'S LAW, AND FRACTAL RABBITS , 2015 .

[70]  Boualem Boashash,et al.  Image fusion-based contrast enhancement , 2012, EURASIP Journal on Image and Video Processing.

[71]  Subrata Rakshit,et al.  Estimation of Structural Information Content in Images , 2006, ACCV.

[72]  Yanqing Li,et al.  No-Reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics , 2017, 2017 2nd International Conference on Multimedia and Image Processing (ICMIP).

[73]  Topi Mäenpää,et al.  The local binary pattern approach to texture analysis - extensions and applications , 2003 .

[74]  Yanguang Chen,et al.  Scaling laws and indications of self-organized criticality in urban systems , 2008 .

[75]  Hong Zhang,et al.  An efficient analytical method for computing the Boltzmann entropy of a landscape gradient , 2018, Trans. GIS.

[76]  Mi Wang,et al.  A Multi-Resolution Blending Considering Changed Regions for Orthoimage Mosaicking , 2016, Remote. Sens..

[77]  Junichi Susaki.  Segmentation of Shadowed Buildings in Dense Urban Areas from Aerial Photographs , 2012, Remote. Sens..

[78]  J. Wesley Roberts,et al.  Assessment of image fusion procedures using entropy, image quality, and multispectral classification , 2008 .

[79]  Zhao Liu,et al.  Low-cost cloud computing solution for geo-information processing , 2016 .

[80]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[81]  B. Jiang,et al.  Ht-Index for Quantifying the Fractal or Scaling Structure of Geographic Features , 2013, 1305.0883.

[82]  C. Achilles,et al.  Evaluation: A Systematic Approach , 1980 .

[83]  Jordi Mestres,et al.  SHED: Shannon Entropy Descriptors from Topological Feature Distributions , 2006, J. Chem. Inf. Model..

[84]  J. Bekenstein,et al.  Information in the holographic universe. , 2003, Scientific American.

[85]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[86]  André Thess.  The Entropy Principle: Thermodynamics for the Unsatisfied , 2011 .

[87]  L. Boltzmann.  Weitere Studien über das Wärmegleichgewicht unter Gasmolekülen , 1970 .

[88]  Ahmed S. Abutaleb,et al.  Automatic thresholding of gray-level pictures using two-dimensional entropy , 1989, Comput. Vis. Graph. Image Process..

[89]  D. Sui.  Tobler's First Law of Geography: A Big Idea for a Small World? , 2004 .

[90]  James C. Gibeaut,et al.  Using UAS Hyperspatial RGB Imagery for Identifying Beach Zones along the South Texas Coast , 2017, Remote. Sens..

[91]  W. Tobler.  A Computer Movie Simulating Urban Growth in the Detroit Region , 1970 .

[92]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[93]  Xiang Li,et al.  A Spatial Entropy‐Based Decision Tree for Classification of Geographical Information , 2006, Trans. GIS.

[94]  Joel Fischer,et al.  Evaluating Practice: Guidelines for the Accountable Professional , 1994 .

[95]  Jun Liu,et al.  Human visual system consistent quality assessment for remote sensing image fusion , 2015 .

[96]  Shuyuan Yang,et al.  Band selection and evaluation with spatial information , 2016 .