Calibration Invariance of the MaxEnt Distribution in the Maximum Entropy Principle

The maximum entropy principle consists of two steps: The first step is to find the distribution which maximizes entropy under given constraints. The second step is to calculate the corresponding thermodynamic quantities. The second part is determined by Lagrange multipliers’ relation to the measurable physical quantities as temperature or Helmholtz free energy/free entropy. We show that for a given MaxEnt distribution, the whole class of entropies and constraints leads to the same distribution but generally different thermodynamics. Two simple classes of transformations that preserve the MaxEnt distributions are studied: The first case is a transform of the entropy to an arbitrary increasing function of that entropy. The second case is the transform of the energetic constraint to a combination of the normalization and energetic constraints. We derive group transformations of the Lagrange multipliers corresponding to these transformations and determine their connections to thermodynamic quantities. For each case, we provide a simple example of this transformation.

[1]  A. M. Scarfone,et al.  Consistency of the structure of Legendre transform in thermodynamics with the Kolmogorov–Nagumo average , 2016, 2206.04414.

[2]  Marek Czachor.  Unifying Aspects of Generalized Calculus , 2020, Entropy.

[3]  Jagdish Mitter,et al.  On Measures of "Useful" Information , 1978, Inf. Control..

[4]  R. Hanel,et al.  A comprehensive classification of complex statistical systems and an axiomatic derivation of their entropy and distribution functions , 2010, 1005.0138.

[5]  R. Johnson,et al.  Properties of cross-entropy minimization , 1981, IEEE Trans. Inf. Theory.

[6]  Tsallis statistics: averages and a physical interpretation of the Lagrange multiplier β , 2000, cond-mat/0006279.

[7]  A. Plastino.  On the universality of thermodynamics' Legendre transform structure , 1997 .

[8]  Y. Klimontovich.  Thermodynamics of Chaotic Systems — An introduction , 1994 .

[9]  Sumiyoshi Abe.  Geometry of escort distributions. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[10]  Stefan Thurner,et al.  Classification of complex systems by their sample-space scaling exponents , 2018, New Journal of Physics.

[11]  P. Jizba,et al.  The world according to R enyi: thermodynamics of multifractal systems , 2002, cond-mat/0207707.

[12]  Kingshuk Ghosh,et al.  Reply to C. Tsallis' "Conceptual Inadequacy of the Shore and Johnson Axioms for Wide Classes of Complex Systems" , 2015, Entropy.

[13]  T S Biró,et al.  Zeroth law compatibility of nonadditive thermodynamics. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[14]  G. Bagci,et al.  Reply to "Comment on 'Rényi entropy yields artificial biases not in the data and incorrect updating due to the finite-size data' ". , 2019, Physical review. E.

[15]  F. Pennini,et al.  Tsallis’ entropy maximization procedure revisited , 2000 .

[16]  Kingshuk Ghosh,et al.  Perspective: Maximum caliber is a general variational principle for dynamical systems. , 2017, The Journal of chemical physics.

[17]  Kingshuk Ghosh,et al.  Nonadditive entropies yield probability distributions with biases not warranted by the data. , 2013, Physical review letters.

[18]  C. Goffman,et al.  Measures on ℝⁿ , 2001 .

[19]  A. Rényi,et al.  Selected papers of Alfréd Rényi , 1976 .

[20]  Jan Havrda,et al.  Quantification method of classification processes. Concept of structural a-entropy , 1967, Kybernetika.

[21]  C. Tsallis.  Possible generalization of Boltzmann-Gibbs statistics , 1988 .

[22]  Constantino Tsallis,et al.  Conceptual Inadequacy of the Shore and Johnson Axioms for Wide Classes of Complex Systems , 2014, Entropy.

[23]  J. Naudts,et al.  Thermostatistics based on Kolmogorov–Nagumo averages: unifying framework for extensive and nonextensive generalizations , 2001, cond-mat/0106324.

[24]  G. Kaniadakis,et al.  Statistical mechanics in the context of special relativity. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[25]  Petr Jizba,et al.  When Shannon and Khinchin meet Shore and Johnson: Equivalence of information theory and statistical inference axiomatics. , 2020, Physical review. E.

[26]  Stefan Thurner,et al.  Introduction to the Theory of Complex Systems , 2018, Oxford Scholarship Online.

[27]  Tsallis ensemble as an exact orthode , 2006, cond-mat/0605089.

[28]  J. Korbel,et al.  Maximum Entropy Principle in Statistical Inference: Case for Non-Shannonian Entropies. , 2018, Physical review letters.

[29]  Jos Uunk,et al.  Can the Maximum Entropy Principle Be Explained as a Consistency Requirement? , 1997 .

[30]  Misusing the entropy maximization in the jungle of generalized entropies , 2016, 1605.01528.

[31]  J. Bercher.  Tsallis distribution as a standard maximum entropy solution with ‘tail’ constraint , 2008 .

[32]  G. Bagci,et al.  Rényi entropy yields artificial biases not in the data and incorrect updating due to the finite-size data. , 2018, Physical review. E.

[33]  C. Tsallis,et al.  The role of constraints within generalized nonextensive statistics , 1998 .

[34]  U. Lucia.  Stationary open systems: A brief review on contemporary theories on irreversibility , 2013 .

[35]  M. Stankovic,et al.  Generalized Shannon–Khinchin axioms and uniqueness theorem for pseudo-additive entropies , 2013, 1311.0323.

[36]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[37]  Rodney W. Johnson,et al.  Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy , 1980, IEEE Trans. Inf. Theory.

[38]  J. Korbel,et al.  Comment on "Rényi entropy yields artificial biases not in the data and incorrect updating due to the finite-size data". , 2019, Physical review. E.

[39]  Pierfrancesco Palazzo,et al.  Hierarchical Structure of Generalized Thermodynamic and Informational Entropy , 2018, Entropy.

[40]  A. M. Scarfone,et al.  Equivalence between four versions of thermostatistics based on strongly pseudoadditive entropies. , 2019, Physical review. E.

[41]  A. R. Plastino,et al.  Tsallis entropy and Jaynes' Information Theory formalism , 1999 .

[42]  Andreas Daffertshofer,et al.  Exact time-dependent solutions of the Renyi Fokker–Planck equation and the Fokker–Planck equations related to the entropies proposed by Sharma and Mittal , 2000 .

[43]  J. Bercher.  On escort distributions, q-gaussians and Fisher information , 2011 .

[44]  Henrik Jeldtoft Jensen,et al.  Universality Classes and Information-Theoretic Measures of Complexity via Group Entropies , 2019, Scientific Reports.

[45]  A. M. Scarfone,et al.  Connections between Tsallis' formalisms employing the standard linear average energy and ones employing the normalized q-average energy , 2004, cond-mat/0410527.

[46]  J. Burg.  THE RELATIONSHIP BETWEEN MAXIMUM ENTROPY SPECTRA AND MAXIMUM LIKELIHOOD SPECTRA , 1972 .