Information Landscape and Flux, Mutual Information Rate Decomposition and Connections to Entropy Production

We explored the dynamics of two interacting information systems. We show that for the Markovian marginal systems, the driving force for information dynamics is determined by both the information landscape and information flux. While the information landscape can be used to construct the driving force to describe the equilibrium time-reversible information system dynamics, the information flux can be used to describe the nonequilibrium time-irreversible behaviors of the information system dynamics. The information flux explicitly breaks the detailed balance and is a direct measure of the degree of the nonequilibrium or time-irreversibility. We further demonstrate that the mutual information rate between the two subsystems can be decomposed into the equilibrium time-reversible and nonequilibrium time-irreversible parts, respectively. This decomposition of the Mutual Information Rate (MIR) corresponds to the information landscape-flux decomposition explicitly when the two subsystems behave as Markov chains. Finally, we uncover the intimate relationship between the nonequilibrium thermodynamics in terms of the entropy production rates and the time-irreversible part of the mutual information rate. We found that this relationship and MIR decomposition still hold for the more general stationary and ergodic cases. We demonstrate the above features with two examples of the bivariate Markov chains.

[1]  Zhedong Zhang,et al.  Landscape, kinetics, paths and statistics of curl flux, coherence, entanglement and energy transfer in non-equilibrium quantum systems , 2015 .

[2]  J. Koski,et al.  On-Chip Maxwell's Demon as an Information-Powered Refrigerator. , 2015, Physical review letters.

[3]  Robert M. Gray,et al.  Mutual information rate, distortion, and quantization in metric spaces , 1980, IEEE Trans. Inf. Theory.

[4]  Haidong Feng,et al.  Potential and flux decomposition for dynamical systems and non-equilibrium thermodynamics: curvature, gauge field, and generalized fluctuation-dissipation theorem. , 2011, The Journal of chemical physics.

[5]  P. Gaspard.  Time-Reversed Dynamical Entropy and Irreversibility in Markovian Random Processes , 2004 .

[6]  Jin Wang.  Landscape and flux theory of non-equilibrium dynamical systems with application to biology , 2015 .

[7]  Brian L. Mark,et al.  Bivariate Markov Processes and Their Estimation , 2013, Found. Trends Signal Process..

[8]  Jordan M. Horowitz,et al.  Thermodynamics with Continuous Information Flow , 2014, 1402.3276.

[9]  Udo Seifert,et al.  Rate of Mutual Information Between Coarse-Grained Non-Markovian Variables , 2013, 1306.1698.

[10]  Gernot Schaller,et al.  Thermodynamics of a physical model implementing a Maxwell demon. , 2012, Physical review letters.

[11]  Jin Wang,et al.  Funneled potential and flux landscapes dictate the stabilities of both the states and the flow: Fission yeast cell cycle , 2017, PLoS Comput. Biol..

[12]  Brian L. Mark,et al.  An EM algorithm for continuous-time bivariate Markov chains , 2011, Comput. Stat. Data Anal..

[13]  Donald F. Towsley,et al.  Continuous-time hidden Markov models for network performance evaluation , 2002, Perform. Evaluation.

[14]  Thomas M. Cover,et al.  Elements of information theory (2. ed.) , 2006 .

[15]  Andre C. Barato,et al.  Stochastic thermodynamics of bipartite systems: transfer entropy inequalities and a Maxwell’s demon interpretation , 2014 .

[16]  Jin Wang,et al.  Potential landscape and flux framework of nonequilibrium networks: Robustness, dissipation, and coherence of biochemical oscillations , 2008, Proceedings of the National Academy of Sciences.

[17]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[18]  M. N. Bera,et al.  Thermodynamics from Information , 2018, 1805.10282.

[19]  Masahito Ueda,et al.  Fluctuation theorem with information exchange: role of correlations in stochastic thermodynamics. , 2012, Physical review letters.

[20]  B. Schmittmann,et al.  Probability currents as principal characteristics in the statistical mechanics of non-equilibrium steady states , 2007 .

[21]  Frank Ball,et al.  Lumpability and marginalisability for continuous-time Markov chains , 1993 .

[22]  Qian Minping,et al.  Circulation for recurrent markov chains , 1982 .

[23]  Zhedong Zhang,et al.  Curl flux, coherence, and population landscape of molecular systems: nonequilibrium quantum steady state, energy (charge) transport, and thermodynamics. , 2014, The Journal of chemical physics.

[24]  J. Schnakenberg.  Network theory of microscopic and macroscopic behavior of master equation systems , 1976 .

[25]  C. Maes,et al.  On the definition of entropy production, via examples , 2000 .

[26]  Jin Wang,et al.  Potential flux landscapes determine the global stability of a Lorenz chaotic attractor under intrinsic fluctuations. , 2012, The Journal of chemical physics.

[27]  P. R. ten Wolde,et al.  Biochemical Machines for the Interconversion of Mutual Information and Work. , 2017, Physical review letters.

[28]  C. Maes,et al.  Canonical structure of dynamical fluctuations in mesoscopic nonequilibrium steady states , 2007, 0705.2344.

[29]  M. Polettini.  Nonequilibrium thermodynamics as a gauge theory , 2011, 1110.0608.