Concentration Theorems for Entropy and Free Energy

AbstractJaynes’s entropy concentration theorem states that, for most words ω1 ...ωN of length N such that $$\mathop \Sigma \limits_{i = 1}^{\rm N} \;f(\omega _i ) \approx vN$$ , empirical frequencies of values of a function f are close to the probabilities that maximize the Shannon entropy given a value v of the mathematical expectation of f. Using the notion of algorithmic entropy, we define the notions of entropy for the Bose and Fermi statistical models of unordered data. New variants of Jaynes’s concentration theorem for these models are proved. We also present some concentration properties for free energy in the case of a nonisolated isothermal system. Exact relations for the algorithmic entropy and free energy at extreme points are obtained. These relations are used to obtain tight bounds on uctuations of energy levels at equilibrium points.