Information Entropy As a Basic Building Block of Complexity Theory

What is information? What role does information entropy play in this information exploding age, especially in understanding emergent behaviors of complex systems? To answer these questions, we discuss the origin of information entropy, the difference between information entropy and thermodynamic entropy, the role of information entropy in complexity theories, including chaos theory and fractal theory, and speculate new fields in which information entropy may play important roles.

[1]  P E Rapp,et al.  Effective normalization of complexity measurements for epoch length and sampling frequency. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[2]  Jianbo Gao,et al.  Entropies of Negative Incomes, Pareto-Distributed Loss, and Financial Crises , 2011, PloS one.

[3]  J B Gao,et al.  Distinguishing chaos from noise by scale-dependent Lyapunov exponent. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[4]  Shor,et al.  Scheme for reducing decoherence in quantum computer memory. , 1995, Physical review. A, Atomic, molecular, and optical physics.

[5]  E. Shaw.  The Schooling of Fishes , 1962 .

[6]  Andrei N. Kolmogorov,et al.  Logical basis for information theory and probability theory , 1968, IEEE Trans. Inf. Theory.

[7]  Wenzhong Shi,et al.  Analysis of Airborne Particulate Matter (PM2.5) over Hong Kong Using Remote Sensing and GIS , 2012, Sensors.

[8]  Jing Hu,et al.  A New Way to Model Nonstationary Sea Clutter , 2009, IEEE Signal Processing Letters.

[9]  Lila L. Gatlin,et al.  Information theory and the living system , 1972 .

[10]  Frank H. Shu,et al.  On the Spiral Structure of Disk Galaxies. , 1964 .

[11]  Rajeev K. Azad,et al.  Information entropy based methods for genome comparison , 2013, SIGBIO.

[12]  I. Good,et al.  The Maximum Entropy Formalism. , 1979 .

[13]  Erik W. Jensen,et al.  EEG complexity as a measure of depth of anesthesia for patients , 2001, IEEE Trans. Biomed. Eng..

[14]  Gerd Sauermann,et al.  Minimal model for sand dunes. , 2002, Physical review letters.

[15]  C. Tsallis,et al.  Statistical-mechanical foundation of the ubiquity of Lévy distributions in Nature. , 1995, Physical review letters.

[16]  Cristian R. Munteanu,et al.  New Markov-Shannon Entropy models to assess connectivity quality in complex networks: from molecular to cellular pathway, Parasite-Host, Neural, Industry, and Legal-Social networks. , 2012, Journal of theoretical biology.

[17]  A. Provenzale,et al.  Finite correlation dimension for stochastic systems with power-law spectra , 1989 .

[18]  R. Feynman.  The Character of Physical Law , 1965 .

[19]  Nithya Ramakrishnan,et al.  Dipole entropy based techniques for segmentation of introns and exons in DNA , 2012 .

[20]  T. Swan,et al.  ECONOMIC GROWTH and CAPITAL ACCUMULATION , 1956 .

[21]  C. Hemelrijk,et al.  Self-organised complex aerial displays of thousands of starlings: a model , 2009, 0908.2677.

[22]  P. Veenhuis.  The Emperor's New Mind: Concerning Computers, Minds, and the Laws of Physics , 1995 .

[23]  Harald Atmanspacher,et al.  A fundamental link between system theory and statistical mechanics , 1987 .

[24]  Jianbo Gao,et al.  On the structures and quantification of recurrence plots , 2000 .

[25]  Ashwin R. Vasavada,et al.  Jovian atmospheric dynamics: an update after Galileo and Cassini , 2005 .

[26]  Carlo Cattani.  Uncertainty and Symmetries in DNA Sequences , 2013 .

[27]  N. Georgescu-Roegen.  The Entropy Law and the Economic Process , 1973 .

[28]  Jing Hu,et al.  Facilitating Joint Chaos and Fractal Analysis of Biosignals through Nonlinear Adaptive Filtering , 2011, PloS one.

[29]  R. Solow.  A Contribution to the Theory of Economic Growth , 1956 .

[30]  Antonio Politi,et al.  Error propagation in extended chaotic systems , 1995, chao-dyn/9504016.

[31]  J. Gillis,et al.  Probability and Related Topics in Physical Sciences , 1960 .

[32]  J. Lebowitz.  Microscopic origins of irreversible macroscopic behavior , 1999 .

[33]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[34]  C. Hemelrijk,et al.  Self-Organized Shape and Frontal Density of Fish Schools , 2008 .

[35]  C. Cattani,et al.  On the Fractal Geometry of DNA by the Binary Image Analysis , 2013, Bulletin of mathematical biology.

[36]  C. Cervato,et al.  How Geoscientists Think and Learn , 2009 .

[37]  C. Villani.  Chapter 2 – A Review of Mathematical Topics in Collisional Kinetic Theory , 2002 .

[38]  Jordan B. Peterson,et al.  Psychological entropy: a framework for understanding uncertainty-related anxiety. , 2012, Psychological review.

[39]  A. Crisanti,et al.  Predictability in the large: an extension of the concept of Lyapunov exponent , 1996, chao-dyn/9606014.

[40]  Jianbo Gao,et al.  Multiscale Analysis of Complex Time Series: Integration of Chaos and Random Fractal Theory, and Beyond , 2007 .

[41]  Madalena Costa,et al.  Multiscale entropy analysis of biological signals. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[42]  J. Gouyet.  Physics and Fractal Structures , 1996 .

[43]  J. B. Gao,et al.  Recurrence Time Statistics for Chaotic Systems and Their Applications , 1999 .

[44]  J. Manyika.  Big data: The next frontier for innovation, competition, and productivity , 2011 .

[45]  Jianbo Gao,et al.  When Can Noise Induce Chaos , 1999 .

[46]  R. Duncan Luce,et al.  Whatever Happened to Information Theory in Psychology? , 2003 .

[47]  P. Gaspard,et al.  Noise, chaos, and (ε,τ)-entropy per unit time , 1993 .

[48]  Vulpiani,et al.  Growth of Noninfinitesimal Perturbations in Turbulence. , 1996, Physical review letters.

[49]  Aleksandr Yakovlevich Khinchin,et al.  Mathematical foundations of information theory , 1959 .

[50]  C. Sparrow.  The Fractal Geometry of Nature , 1984 .

[51]  A. Bertozzi,et al.  Self-propelled particles with soft-core interactions: patterns, stability, and collapse. , 2006, Physical review letters.

[52]  Jianbo Gao,et al.  Complexity measures of brain wave dynamics , 2011, Cognitive Neurodynamics.

[53]  Charlotte K. Hemelrijk,et al.  Some Causes of the Variable Shape of Flocks of Birds , 2011, PloS one.

[54]  P. Grassberger,et al.  Estimation of the Kolmogorov entropy from a chaotic signal , 1983 .

[55]  V. Arnold.  Mathematical Methods of Classical Mechanics , 1974 .

[56]  Jürgen Mimkes,et al.  Stokes integral of economic growth , 2010 .

[57]  C. Tsallis.  Possible generalization of Boltzmann-Gibbs statistics , 1988 .

[58]  Jing Hu,et al.  Analysis of Biomedical Signals by the Lempel-Ziv Complexity: the Effect of Finite Data Size , 2006, IEEE Transactions on Biomedical Engineering.

[59]  Kenneth Falconer,et al.  Fractal Geometry: Mathematical Foundations and Applications , 1990 .

[60]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[61]  V. Roychowdhury,et al.  Assessment of long-range correlation in time series: how to avoid pitfalls. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[62]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[63]  A. Kolmogorov.  Three approaches to the quantitative definition of information , 1968 .

[64]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[65]  Abraham Lempel,et al.  On the Complexity of Finite Sequences , 1976, IEEE Trans. Inf. Theory.

[66]  David Loewenstern,et al.  Significantly Lower Entropy Estimates for Natural DNA Sequences , 1999, J. Comput. Biol..

[67]  Jianbo Gao,et al.  Quantifying dynamical predictability: the pseudo-ensemble approach , 2009 .

[68]  Craig W. Reynolds.  Flocks, herds, and schools: a distributed behavioral model , 1998 .

[69]  M. C. Soriano,et al.  Distinguishing chaotic and stochastic dynamics from time series by using a multiscale symbolic approach. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[70]  Frank Emmert-Streib,et al.  Statistic Complexity: Combining Kolmogorov Complexity with an Ensemble Approach , 2010, PloS one.

[71]  Jianbo Gao,et al.  Revised Papers , 2022 .

[72]  Jianbo Gao,et al.  Detecting low-dimensional chaos by the "noise titration" technique: Possible problems and remedies , 2012 .

[73]  Richard W. Hamming,et al.  The Art of Probability for Scientists and Engineers , 1991 .

[74]  Carlo Cattani,et al.  On the Existence of Wavelet Symmetries in Archaea DNA , 2011, Comput. Math. Methods Medicine.

[75]  Radhakrishnan Nagarajan,et al.  Quantifying physiological data with Lempel-Ziv complexity-certain issues , 2002, IEEE Transactions on Biomedical Engineering.

[76]  J. McCauley.  Thermodynamic analogies in economics and finance: instability of markets , 2003 .

[77]  Ivan Erill,et al.  I NFORMATION T HEORY AND B IOLOGICAL S EQUENCES : I NSIGHTS FROM AN E VOLUTIONARY P ERSPECTIVE , 2012 .

[78]  Jianbo Gao,et al.  Multiscale analysis of economic time series by scale-dependent Lyapunov exponent , 2013 .

[79]  Benoit B. Mandelbrot,et al.  Fractal Geometry of Nature , 1984 .

[80]  J. Crutchfield,et al.  Measures of statistical complexity: Why? , 1998 .

[81]  Ilya Prigogine,et al.  From Being To Becoming , 1980 .

[82]  Debra J Searles,et al.  Experimental demonstration of violations of the second law of thermodynamics for small systems and short time scales. , 2002, Physical review letters.

[83]  A. Provenzale,et al.  Convergence of the K 2 entropy for random noises with power law spectra , 1991 .

[84]  J. Crutchfield,et al.  Structural information in two-dimensional patterns: entropy convergence and excess entropy. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[85]  I. Good,et al.  The Maximum Entropy Formalism. , 1979 .

[86]  E. Schrödinger,et al.  What is life? : the physical aspect of the living cell , 1946 .