From Rate Distortion Theory to Metric Mean Dimension: Variational Principle

The purpose of this paper is to point out a new connection between information theory and dynamical systems. In the information theory side, we consider rate distortion theory, which studies lossy data compression of stochastic processes under distortion constraints. In the dynamical systems side, we consider mean dimension theory, which studies how many parameters per iterate we need to describe a dynamical system. The main results are new variational principles connecting rate distortion function to metric mean dimension.

[1]  G. Elliott,et al.  The C∗-algebra of a minimal homeomorphism of zero mean dimension , 2014, 1406.2382.

[2]  A. Velozo,et al.  Rate distortion theory, metric mean dimension and measure theoretic entropy , 2017, 1707.05762.

[3]  Yonatan Gutman.  Embedding topological dynamical systems with periodic points in cubical shifts , 2015, Ergodic Theory and Dynamical Systems.

[4]  Hanfeng Li,et al.  Mean dimension, mean rank, and von Neumann–Lück rank , 2013, Journal für die reine und angewandte Mathematik (Crelles Journal).

[5]  Yonatan Gutman.  Mean dimension and Jaworski‐type theorems , 2012, 1208.5248.

[6]  M. Tsukamoto.  Mean dimension of the dynamical system of Brody curves , 2014, 1410.1143.

[7]  David L. Neuhoff,et al.  New results on coding of stationary nonergodic sources , 1979, IEEE Trans. Inf. Theory.

[8]  E. Lindenstrauss,et al.  Mean dimension and an embedding problem: An example , 2014 .

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  Amir Dembo,et al.  The rate-distortion dimension of sets and measures , 1994, IEEE Trans. Inf. Theory.

[11]  Yihong Wu,et al.  Rényi Information Dimension: Fundamental Limits of Almost Lossless Analog Compression , 2010, IEEE Transactions on Information Theory.

[12]  P. Walters.  A VARIATIONAL PRINCIPLE FOR THE PRESSURE OF CONTINUOUS TRANSFORMATIONS. , 1975 .

[13]  L. Carleson,et al.  The Collected Works of Arne Beurling , 1989 .

[14]  A. Rényi.  On the dimension and entropy of probability distributions , 1959 .

[15]  Yonatan Gutman,et al.  Embedding minimal dynamical systems into Hilbert cubes , 2015, Inventiones mathematicae.

[16]  M. Tsukamoto.  Large dynamics of Yang–Mills theory: mean dimension formula , 2014, 1407.2058.

[17]  Yonina C. Eldar.  Sampling Theory: Beyond Bandlimited Systems , 2015 .

[18]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[19]  David Ruelle,et al.  Ergodic Theory of Axiom a Diffeomorphisms , 1975 .

[20]  T. Goodman,et al.  Relating Topological Entropy and Measure Entropy , 1971 .

[21]  E. Lindenstrauss.  Mean dimension, small entropy factors and an embedding theorem , 1999 .

[22]  Me Misiurewicz,et al.  A short proof of the variational principle for a ZN+ action on a compact space , 1975 .

[23]  Yonatan Gutman.  Embedding ℤk-actions in cubical shifts and ℤk-symbolic extensions , 2010, Ergodic Theory and Dynamical Systems.

[24]  Benjamin Weiss,et al.  Mean topological dimension , 2000 .

[25]  R. Gray.  Entropy and Information Theory , 1990, Springer New York.

[26]  B. Costa.  Deux exemples sur la dimension moyenne d'un espace de courbes de Brody , 2011, 1110.6082.

[27]  C. Villani.  Optimal Transport: Old and New , 2008 .

[28]  Aaron D. Wyner,et al.  Coding Theorems for a Discrete Source With a Fidelity CriterionInstitute of Radio Engineers, International Convention Record, vol. 7, 1959. , 1993 .

[29]  H. Vincent Poor,et al.  Rate-distortion dimension of stochastic processes , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[30]  Misha Gromov,et al.  Topological Invariants of Dynamical Systems and Spaces of Holomorphic Maps: I , 1999 .

[31]  A. Kolmogorov,et al.  Entropy and "-capacity of sets in func-tional spaces , 1961 .

[32]  L. Ambrosio,et al.  Gradient Flows: In Metric Spaces and in the Space of Probability Measures , 2005 .

[33]  D. Ornstein.  Bernoulli shifts with the same entropy are isomorphic , 1970 .

[34]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .

[35]  Chapter 10 – On the Interplay between Measurable and Topological Dynamics , 2004, math/0408328.

[36]  Michelle Effros,et al.  Variable-rate source coding theorems for stationary nonergodic sources , 1994, IEEE Trans. Inf. Theory.

[37]  Hanfeng Li.  Sofic mean dimension , 2011, 1105.0140.

[38]  Shinichiroh Matsuo,et al.  Brody curves and mean dimension , 2011, 1110.6014.

[39]  Benjamin Weiss,et al.  Entropy is the Only Finitely Observable Invariant , 2006 .

[40]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[41]  Y. Katznelson,et al.  ON UNIQUE ERGODICITY , 2005 .

[42]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[43]  Roy L. Adler,et al.  Topological entropy , 2008, Scholarpedia.