General H-theorem and Entropies that Violate the Second Law

H-theorem states that the entropy production is nonnegative and, therefore, the entropy of a closed system should monotonically change in time. In information processing, the entropy production is positive for random transformation of signals (the information processing lemma). Originally, the H-theorem and the information processing lemma were proved for the classical Boltzmann-Gibbs-Shannon entropy and for the correspondent divergence (the relative entropy). Many new entropies and divergences have been proposed during last decades and for all of them the H-theorem is needed. This note proposes a simple and general criterion to check whether the H-theorem is valid for a convex divergence H and demonstrates that some of the popular divergences obey no H-theorem. We consider systems with n states Ai that obey first order kinetics (master equation). A convex function H is a Lyapunov function for all master equations with given equilibrium if and only if its conditional minima properly describe the equilibria of pair transitions Ai ⇌ Aj . This theorem does not depend on the principle of detailed balance and is valid for general Markov kinetics. Elementary analysis of pair equilibria demonstrate that the popular Bregman divergences like Euclidian distance or Itakura-Saito distance in the space of distribution cannot be the universal Lyapunov functions for the first-order kinetics and can increase in Markov processes. Therefore, they violate the second law and the information processing lemma. In particular, for these measures of information (divergences) random manipulation with data may add information to data. The main results are extended to nonlinear generalized mass action law kinetic equations.

[1]  G. Crooks On Measures of Entropy and Information , 2015 .

[2]  Ezra Miller,et al.  A Geometric Approach to the Global Attractor Conjecture , 2013, SIAM J. Appl. Dyn. Syst..

[3]  Alexander N. Gorban Maxallent: Maximizers of all entropies and uncertainty of uncertainty , 2013, Comput. Math. Appl..

[4]  Alexander N Gorban,et al.  Thermodynamics in the limit of irreversible reactions , 2012, Physica A: Statistical Mechanics and its Applications.

[5]  Alexander N Gorban,et al.  Local equivalence of reversible and general Markov kinetics , 2013 .

[6]  Alexander N. Gorban,et al.  Thermodynamic Tree: The Space of Admissible Paths , 2012, SIAM J. Appl. Dyn. Syst..

[7]  Fedor Nazarov,et al.  Persistence and Permanence of Mass-Action and Power-Law Dynamical Systems , 2010, SIAM J. Appl. Math..

[8]  Miroslav Grmela,et al.  Fluctuations in extended mass-action-law dynamics , 2012 .

[9]  V. Giovangigli,et al.  Supercritical fluid thermodynamics from equations of state , 2012 .

[10]  Imre Csiszár,et al.  Generalized minimizers of convex integral functionals, Bregman distance, Pythagorean identities , 2012, Kybernetika.

[11]  Friedrich Hasenöhrl,et al.  Neuer Beweis zweier Sätze über das Wärmegleichgewicht unter mehratomigen Gasmolekülen , 2012 .

[12]  Z. Tuza,et al.  Finding weakly reversible realizations of chemical reaction networks using optimization , 2011, 1103.4741.

[13]  Alexander N Gorban,et al.  Extended detailed balance for systems with irreversible reactions , 2011, 1101.5280.

[14]  G. Szederkényi,et al.  Finding complex balanced and detailed balanced realizations of chemical reaction networks , 2010, 1010.4477.

[15]  Muhammad Shahzad,et al.  The Michaelis-Menten-Stueckelberg Theorem , 2010, Entropy.

[16]  Alexander N Gorban Kinetic path summation, multi-sheeted extension of master equation, and evaluation of ergodicity coefficient , 2010, 1006.4128.

[17]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[18]  V. Kolokoltsov Nonlinear Markov Processes and Kinetic Equations , 2010 .

[19]  Andrzej Cichocki,et al.  Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities , 2010, Entropy.

[20]  Katalin M. Hangos Engineering Model Reduction and Entropy-based Lyapunov Functions in Chemical Reaction Kinetics , 2010, Entropy.

[21]  Alexander N. Gorban,et al.  Entropy: The Markov Ordering Approach , 2010, Entropy.

[22]  Shun-ichi Amari,et al.  Divergence, Optimization and Geometry , 2009, ICONIP.

[23]  Sean P. Meyn Control Techniques for Complex Networks: Workload , 2007 .

[24]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[25]  gelehrter Gesellschaften Sitzungsberichte der Kaiserlichen Akademie der Wissenschaften in Wien , 2006, Naturwissenschaften.

[26]  Alexander N. Gorban,et al.  Thermodynamic Equilibria and Extrema: Analysis of Attainability regions and Partial Equilibrium , 2006 .

[27]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[28]  A. Gorban,et al.  Invariant Manifolds for Physical and Chemical Kinetics , 2005 .

[29]  JOEL E. COHEN,et al.  Majorization , Monotonicity of Relative Entropy , and Stochastic Matrices , 2005 .

[30]  Alexander N Gorban,et al.  Uniqueness of thermodynamic projector and kinetic basis of molecular individualism , 2003, cond-mat/0309638.

[31]  Alexander N Gorban,et al.  Legendre integrators, post-processing and quasiequilibrium , 2003, cond-mat/0308488.

[32]  J. Lebowitz,et al.  On the (Boltzmann) entropy of non-equilibrium systems , 2003, cond-mat/0304251.

[33]  Pavel Gorban Monotonically equivalent entropies and solution of additivity equation , 2003 .

[34]  Iliya V. Karlin,et al.  Method of invariant manifold for chemical kinetics , 2003 .

[35]  阿部 純義,et al.  Nonextensive statistical mechanics and its applications , 2001 .

[36]  On Directional Convexity , 2001 .

[37]  I. Csiszár,et al.  Information projections revisited , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).

[38]  Miroslav Grmela,et al.  Dynamics and thermodynamics of complex fluids. I. Development of a general formalism , 1997 .

[39]  Uriel G. Rothblum,et al.  Directional-Quasi-Convexity, Asymmetric Schur-Convexity and Optimality of Consecutive Partitions , 1996, Math. Oper. Res..

[40]  W. Ebeling Stochastic Processes in Physics and Chemistry , 1995 .

[41]  S. Kalpazidou Cycle representations of Markov processes , 1995 .

[42]  Domingo Morales,et al.  A summary on entropy statistics , 1995, Kybernetika.

[43]  Joel L. Lebowitz,et al.  Boltzmann's Entropy and Time's Arrow , 1993 .

[44]  Joel E. Cohen,et al.  Relative entropy under mappings by stochastic matrices , 1993 .

[45]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[46]  Richard G. Compton,et al.  Kinetic models of catalytic reactions , 1991 .

[47]  C. Tsallis Possible generalization of Boltzmann-Gibbs statistics , 1988 .

[48]  Christian Zylka A note on the attainability of states by equalizing processes , 1985 .

[49]  R. Callen,et al.  Thermodynamics and an Introduction to Thermostatistics, 2nd Edition , 1985 .

[50]  V. N. Orlov,et al.  The macrodynamics of open systems and the variational principle of the local potential—I , 1984 .

[51]  Timothy R. C. Read,et al.  Multinomial goodness-of-fit tests , 1984 .

[52]  N. Kampen,et al.  Stochastic processes in physics and chemistry , 1981 .

[53]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .

[54]  Alexander N. Gorban Invariant sets for kinetic equations , 1979 .

[55]  Ingram Olkin,et al.  Inequalities: Theory of Majorization and Its Application , 1979 .

[56]  C. Bamford,et al.  Comprehensive Chemical Kinetics , 1976 .

[57]  M. Feinberg,et al.  Dynamics of open chemical systems and the algebraic structure of the underlying reaction network , 1974 .

[58]  J. Burg THE RELATIONSHIP BETWEEN MAXIMUM ENTROPY SPECTRA AND MAXIMUM LIKELIHOOD SPECTRA , 1972 .

[59]  R. Jackson,et al.  General mass action kinetics , 1972 .

[60]  Martin Feinberg,et al.  On chemical kinetics of a certain class , 1972 .

[61]  Harvey J. Greenberg,et al.  A Review of Quasi-Convex Functions , 1971, Oper. Res..

[62]  L. Bregman The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[63]  J. Ponstein,et al.  Seven kinds of convexity , 1967 .

[64]  E. T. Jaynes Gibbs vs Boltzmann Entropies , 1965 .

[65]  T. Morimoto Markov Processes and the H -Theorem , 1963 .

[66]  H. Callen Thermodynamics and an Introduction to Thermostatistics , 1988 .