Entropy-Maximising Diffusions Satisfy a Parallel Transport Law

. We show that the principle of maximum entropy, a variational method ap-pearing in statistical inference, statistical physics, and the analysis of stochastic dynamical systems, admits a geometric description from gauge theory. Using the connection on a principal G -bundle, the gradient flow maximising entropy is written in terms of constraint functions which interact with the dynamics of the probabilistic degrees of freedom of a diffusion process. This allows us to describe the point of maximum entropy as parallel transport over the state space. In particular, it is proven that the solubility of the stationary Fokker–Planck equation corresponds to the existence of parallel transport in a particular associated vector bundle, extending classic results due to Jordan–Kinderlehrer–Otto and Markowich–Villani. A reinterpretation of splitting results in stochastic dynamical systems is also suggested. Beyond stochastic analysis, we are able to indicate a collection of geometric structures surrounding energy-based inference in statistics

[1]  Lancelot Da Costa,et al.  Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents , 2022, ArXiv.

[2]  Mark Girolami,et al.  A Unifying and Canonical Description of Measure-Preserving Diffusions , 2021, 2105.02845.

[3]  Terrence J Sejnowski,et al.  The unreasonable effectiveness of deep learning in artificial intelligence , 2020, Proceedings of the National Academy of Sciences.

[4]  Karl J. Friston.  A free energy principle for a particular physics , 2019, 1906.10184.

[5]  E. Witten.  Symmetry and emergence , 2017, Nature Physics.

[6]  Karl J. Friston,et al.  Towards a Neuronal Gauge Theory , 2016, PLoS biology.

[7]  Max Tegmark,et al.  Why Does Deep and Cheap Learning Work So Well? , 2016, Journal of Statistical Physics.

[8]  Kingshuk Ghosh,et al.  Reply to C. Tsallis' "Conceptual Inadequacy of the Shore and Johnson Axioms for Wide Classes of Complex Systems" , 2015, Entropy.

[9]  Kingshuk Ghosh,et al.  Nonadditive entropies yield probability distributions with biases not warranted by the data. , 2013, Physical review letters.

[10]  K. Dill,et al.  Principles of maximum entropy and maximum caliber in statistical physics , 2013 .

[11]  H. Qian.  A decomposition of irreversible diffusion processes without detailed balance , 2012, 1204.6496.

[12]  M. Polettini.  Nonequilibrium thermodynamics as a gauge theory , 2011, 1110.0608.

[13]  John C. Baez,et al.  An invitation to higher gauge theory , 2010, 1003.4485.

[14]  K. Elworthy,et al.  Equivariant Diffusions on Principal Bundles , 2019, 1911.08224.

[15]  J. Lott.  Some Geometric Calculations on Wasserstein Space , 2006, math/0612562.

[16]  Fu Jie Huang,et al.  A Tutorial on Energy-Based Learning , 2006 .

[17]  J. Baez,et al.  Higher gauge theory , 2005, math/0511710.

[18]  P Ao,et al.  LETTER TO THE EDITOR: Potential in stochastic differential equations: novel construction , 2004 .

[19]  C. Villani,et al.  ON THE TREND TO EQUILIBRIUM FOR THE FOKKER-PLANCK EQUATION : AN INTERPLAY BETWEEN PHYSICS AND FUNCTIONAL ANALYSIS , 2004 .

[20]  V. Rubakov.  Classical Theory of Gauge Fields , 2002 .

[21]  Shun-ichi Amari,et al.  Information geometry on hierarchy of probability distributions , 2001, IEEE Trans. Inf. Theory.

[22]  K. Elworthy,et al.  Concerning the geometry of stochastic differential equations and stochastic flows , 2019, 1911.07941.

[23]  D. Kinderlehrer,et al.  THE VARIATIONAL FORMULATION OF THE FOKKER-PLANCK EQUATION , 1996 .

[24]  Yann Le Cun,et al.  A Theoretical Framework for Back-Propagation , 1988 .

[25]  Kiyosi Itô,et al.  The Brownian Motion and Tensor Fields on Riemannian Manifold , 1987 .

[26]  Johannes Jahn,et al.  Duality in vector optimization , 1983, Math. Program..

[27]  P. Goddard.  Gauge Theory and Variational Principles: Global Analysis, Pure and Applied Series A, Vol 1 , 1982 .

[28]  D. Bleecker,et al.  Gauge theory and variational principles , 1981 .

[29]  Robert Graham,et al.  Covariant formulation of non-equilibrium statistical thermodynamics , 1977 .

[30]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .