On a 2-Relative Entropy

We construct a 2-categorical extension of the relative entropy functor of Baez and Fritz, and show that our construction is functorial with respect to vertical morphisms. Moreover, we show such a ‘2-relative entropy’ satisfies natural 2-categorial analogues of convex linearity, vanishing under optimal hypotheses, and lower semicontinuity. While relative entropy is a relative measure of information between probability distributions, we view our construction as a relative measure of information between channels.

[1]  James Fullwood.  An axiomatic characterization of mutual information , 2021, ArXiv.

[2]  Tom Leinster,et al.  A Characterization of Entropy in Terms of Information Loss , 2011, Entropy.

[3]  Robert W. Spekkens,et al.  A mathematical theory of resources , 2014, Inf. Comput..

[4]  F. Segal,et al.  A CHARACTERIZATION OF FIBRANT SEGAL CATEGORIES , 2006, math/0603400.

[5]  Dominic R. Verity,et al.  ∞-Categories for the Working Mathematician , 2018 .

[6]  T. Fritz.  A synthetic approach to Markov kernels, conditional independence, and theorems on sufficient statistics , 2019, ArXiv.

[7]  Bart Jacobs,et al.  Disintegration and Bayesian inversion via string diagrams , 2017, Mathematical Structures in Computer Science.

[8]  Entropy and Diversity: The Axiomatic Approach , 2020, ArXiv.

[9]  John C. Baez,et al.  A Bayesian Characterization of Relative Entropy , 2014, ArXiv.

[10]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[11]  Brendan Fong,et al.  Causal Theories: A Categorical Perspective on Bayesian Networks , 2013, 1301.6201.

[12]  Arthur J. Parzygnat.  A functorial characterization of von Neumann entropy , 2020, ArXiv.

[13]  Tom Leinster,et al.  A short characterization of relative entropy , 2017, Journal of Mathematical Physics.

[14]  James Fullwood,et al.  The Information Loss of a Stochastic Map , 2021, Entropy.

[15]  P. V. Golubtsov.  Information Transformers: Category-Theoretical Structure, Informativeness, Decision-Making Problems , 2004 .