Entropy in deterministic and random systems

The purpose of this course is to introduce some notions of entropy: entropy in information theory, entropy of a curve and topological and measure-theoretic entropies. For these last two notions, we will consider in particular dynamical symbolical systems in order to present some measures of disorder for sequences. We will allude then to the problem of classification of sequences with respect to their spectral properties thanks to the entropy. For this purpose, we will introduce the sequence of block entropies for sequences taking their values in a finite alphabet: we will then compute explicitely the block frequencies (or in other words, the measure of the associated dynamical system) for some examples of automatic sequences (Prouhet-Thue-Morse, paperfolding and Rudin-Shapiro sequences) and for Sturmian sequences (these are the sequences with minimal complexity among all non-ultimately periodic sequences; in particular, we will consider some generalized Fibonacci sequences). But, in order to understand the intuitive meaning of the notions of topological and measure-theoretic entropies, we will begin by defining the Shannon entropy of an experiment.

[1]  R. Hartley.  Transmission of information , 1928 .

[2]  G. Rauzy.  Nombres normaux et processus déterministes , 1976 .

[3]  A. Shiryayev.  New Metric Invariant of Transitive Dynamical Systems and Automorphisms of Lebesgue Spaces , 1993 .

[4]  Wolfgang Krieger,et al.  On entropy and generators of measure-preserving transformations , 1970 .

[5]  J. Neumann,et al.  Operator Methods in Classical Mechanics, II , 1942 .

[6]  Ya. Sinai,et al.  FLOWS WITH FINITE ENTROPY , 1959 .

[7]  B. Burrows,et al.  Measurement of disorder in non-periodic sequences , 1991 .

[8]  N. B. Slater,et al.  Gaps and steps for the sequence nθ mod 1 , 1967, Mathematical Proceedings of the Cambridge Philosophical Society.

[9]  Jean-Paul Allouche,et al.  Generalized Rudin-Shapiro sequences , 1991 .

[10]  Gérard Rauzy,et al.  Représentation géométrique de suites de complexité $2n+1$ , 1991 .

[11]  G. A. Hedlund,et al.  Symbolic Dynamics II. Sturmian Trajectories , 1940 .

[12]  Valérie Berthé,et al.  Conditional entropy of some automatic sequences , 1994 .

[13]  Jean-Paul Allouche,et al.  Complexité des suites de Rudin-Shapiro généralisées , 1993 .

[14]  William Parry,et al.  Topics in Ergodic Theory , 1981 .

[15]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[16]  Jean-Paul Allouche,et al.  Sur la complexite des suites in nies , 1994 .

[17]  Valérie Berthé,et al.  Frequencies of Sturmian series factors , 1996 .

[18]  D. Ornstein.  Bernoulli shifts with the same entropy are isomorphic , 1970 .

[19]  P. Billingsley,et al.  Ergodic theory and information , 1966 .

[20]  Norbert Wiener,et al.  Cybernetics: Control and Communication in the Animal and the Machine. , 1949 .

[21]  D. Newton,et al.  ERGODIC THEORY (Cambridge Studies in Advanced Mathematics, 2) , 1984 .