Entropy: a guide for the perplexed

Entropy is ubiquitous in physics, and it plays important roles in numerous other disciplines ranging from logic and statistics to biology and economics. However, a closer look reveals a complicated picture: entropy is defined differently in different contexts, and even within the same domain different notions of entropy are at work. Some of these are defined in terms of probabilities, others are not. The aim of this chapter is to arrive at an understanding of some of the most important notions of entropy and to clarify the relations between them, After setting the stage by introducing the thermodynamic entropy (Section 2), we discuss notions of entropy in information theory (Section 3), statistical mechanics (Section 4), dynamical systems theory (Section 5) and fractal geometry (Section 6).

[1]  R. Frigg,et al.  Typicality and the Approach to Equilibrium in Boltzmannian Statistical Mechanics , 2009, Philosophy of Science.

[2]  Charlotte Werndl,et al.  Are deterministic descriptions and indeterministic descriptions observationally equivalent , 2009, 1310.1615.

[3]  C. Werndl.  DETERMINISTIC VERSUS INDETERMINISTIC DESCRIPTIONS: NOT THAT DIFFERENT AFTER ALL? , 2009 .

[4]  C. Werndl.  What Are the New Implications of Chaos for Unpredictability? , 2009, The British Journal for the Philosophy of Science.

[5]  R. Frigg,et al.  A Field Guide to Recent Work on the Foundations of Statistical Mechanics. , 2008, 0804.0399.

[6]  D. A. Lavis,et al.  Boltzmann, Gibbs, and the Concept of Equilibrium , 2007, Philosophia Scientiæ.

[7]  Roman Frigg,et al.  The ergodic hierarchy, randomness and Hamiltonian chaos ☆ , 2006 .

[8]  A. G. Bashkirov.  Renyi entropy as a statistical entropy for complex systems , 2006 .

[9]  Roman Frigg,et al.  Chaos and randomness: An equivalence proof of a generalized version of the Shannon entropy and the Kolmogorov–Sinai entropy for Hamiltonian dynamical systems , 2006 .

[10]  M. Hemmo,et al.  Von Neumann’s Entropy Does Not Correspond to Thermodynamic Entropy* , 2006, Philosophy of Science.

[11]  Eulalia Szmidt,et al.  Uncertainty and Information: Foundations of Generalized Information Theory , 2005 .

[12]  R. Sorkin.  Ten theses on black hole entropy , 2005, hep-th/0504037.

[13]  Roman Frigg,et al.  In What Sense is the Kolmogorov-Sinai Entropy a Measure for Chaotic Behaviour?—Bridging the Gap Between Dynamical Systems Theory and Communication Theory , 2004, The British Journal for the Philosophy of Science.

[14]  D. H. Mellor,et al.  Probability: A Philosophical Introduction , 2004 .

[15]  D. A. Lavis,et al.  The Spin-Echo System Reconsidered , 2003, cond-mat/0311527.

[16]  P. Jizba,et al.  The world according to R enyi: thermodynamics of multifractal systems , 2002, cond-mat/0207707.

[17]  Chuang Liu,et al.  The Logic of Thermostatistical Physics , 2001 .

[18]  S. Goldstein.  Boltzmann's Approach to Statistical Mechanics , 2001, cond-mat/0105242.

[19]  J. Uffink.  Bluff Your Way in the Second Law of Thermodynamics , 2000, cond-mat/0005327.

[20]  J. Lebowitz.  Statistical Mechanics: A Selective Review of Two Central Issues , 1999, math-ph/0010018.

[21]  Robert W. Batterman,et al.  Chaos and algorithmic complexity , 1996 .

[22]  C. Howson.  Theories of Probability , 1995 .

[23]  Orly Shenker,et al.  Fractal geometry is not the geometry of nature , 1994 .

[24]  Lawrence Sklar,et al.  Physics and Chance: Philosophical Issues in the Foundations of Statistical Mechanics , 1993 .

[25]  Shunsuke Ihara,et al.  Information theory - for continuous systems , 1993 .

[26]  C. Beck,et al.  Thermodynamics of chaotic systems: References , 1993 .

[27]  Kenneth Falconer,et al.  Fractal Geometry: Mathematical Foundations and Applications , 1990 .

[28]  C. Tsallis.  Possible generalization of Boltzmann-Gibbs statistics , 1988 .

[29]  R. Mañé,et al.  Ergodic Theory and Differentiable Dynamics , 1986 .

[30]  D. Ruelle,et al.  Ergodic theory of chaos and strange attractors , 1985 .

[31]  Robert Shaw,et al.  The Dripping Faucet As A Model Chaotic System , 1984 .

[32]  Benoit B. Mandelbrot,et al.  Fractal Geometry of Nature , 1984 .

[33]  A. Wehrl.  General properties of entropy , 1978 .

[34]  Wilson A. Sutherland,et al.  Introduction to Metric and Topological Spaces , 1975 .

[35]  John Hawkes,et al.  Hausdorff Measure, Entropy, and the Independence of Small Sets , 1974 .

[36]  L. Goodwyn.  Comparing Topological Entropy with Measure-Theoretic Entropy , 1972 .

[37]  R. Bowen.  Periodic points and measures for Axiom $A$ diffeomorphisms , 1971 .

[38]  G. Pólya.  Patterns of plausible inference , 1970 .

[39]  E. T. Jaynes.  Gibbs vs Boltzmann Entropies , 1965 .

[40]  Harold Grad,et al.  The many faces of entropy , 1961 .

[41]  A. B. Pippard,et al.  The Elements of Classical Thermodynamics , 1958 .

[42]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[43]  Claude E. Shannon,et al.  The mathematical theory of communication , 1950 .

[44]  R. Tolman,et al.  The Principles of Statistical Mechanics. By R. C. Tolman. Pp. xix, 661. 40s. 1938. International series of monographs on physics. (Oxford) , 1939, The Mathematical Gazette.

[45]  R. Hartley.  Transmission of information , 1928 .

[46]  J. Gibbs.  Elementary Principles in Statistical Mechanics , 1902 .

[47]  R. Frigg,et al.  Why Typicality Does Not Explain the Approach to Equilibrium , 2011 .

[48]  R. Frigg,et al.  Determinism and Chance from a Humean Perspective , 2010 .

[49]  J. Honerkamp,et al.  Thermodynamik und Statistische Mechanik , 2010 .

[50]  Arno Berger,et al.  Chaos and Chance: An Introduction to Stochastic Aspects of Dynamics , 2001 .

[51]  Jos Ufnk,et al.  Bluff your way in the Second Law of Thermodynamics , 2001 .

[52]  D. Gillies.  Philosophical Theories of Probability , 2000 .

[53]  A. Shiryayev.  New Metric Invariant of Transitive Dynamical Systems and Automorphisms of Lebesgue Spaces , 1993 .

[54]  Lawrence Sklar,et al.  Physics and Chance , 1993 .

[55]  E. Ott.  Chaos in Dynamical Systems: Contents , 1993 .

[56]  G. A. Edgar.  Measure, Topology, and Fractal Geometry , 1990 .

[57]  C. Howson,et al.  Scientific Reasoning: The Bayesian Approach , 1989 .

[58]  E. T. Jaynes,et al.  Papers on probability, statistics and statistical physics , 1983 .

[59]  John Earman,et al.  Laplacian Determinism, or Is This Any Way to Run a Universe? , 1971 .

[60]  Mary Hesse,et al.  Models and analogies in science , 1970 .

[61]  G. Wyllie.  Elementary Statistical Mechanics , 1970 .

[62]  O. Penrose.  Foundations of statistical mechanics , 1969 .

[63]  H. Reiss.  Methods of thermodynamics , 1965 .

[64]  T. Austin,et al.  Ergodic Theory , 2022 .

[65]  A. Kolmogorov,et al.  Entropy and "-capacity of sets in func-tional spaces , 1961 .

[66]  A. Rényi.  On Measures of Entropy and Information , 1961 .

[67]  R. Lindsay,et al.  The Conceptual Foundations of the Statistical Approach in Mechanics , 1959 .