Optimal Renormalization Group Transformation from Information Theory

The connections between information theory, statistical physics and quantum field theory have been the focus of renewed attention. In particular, the renormalization group (RG) has been explored from this perspective. Recently, a variational algorithm employing machine learning tools to identify the relevant degrees of freedom of a statistical system by maximizing an information-theoretic quantity, the real-space mutual information (RSMI), was proposed for real-space RG. Here we investigate analytically the RG coarse-graining procedure and the renormalized Hamiltonian, which the RSMI algorithm defines. By a combination of general arguments, exact calculations and toy models we show that the RSMI coarse-graining is optimal in a sense we define. In particular, a perfect RSMI coarse-graining generically does not increase the range of a short-ranged Hamiltonian, in any dimension. For the case of the 1D Ising model we perturbatively derive the dependence of the coefficients of the renormalized Hamiltonian on the real-space mutual information retained by a generic coarse-graining procedure. We also study the dependence of the optimal coarse-graining on the prior constraints on the number and type of coarse-grained variables. We construct toy models illustrating our findings.