ENTROPY, TRANSINFORMATION AND WORD DISTRIBUTION OF INFORMATION-CARRYING SEQUENCES

We investigate correlations in information carriers, e.g. texts and pieces of music, which are represented by strings of letters. For information carrying strings generated by one source (i.e. a novel or a piece of music) we find correlations on many length scales. The word distribution, the higher order entropy and the transinformation are calculated. The analogy to strings generated through symbolic dynamics by nonlinear systems in critical states is discussed.

[1]  B. McMillan.  The Basic Theorems of Information Theory , 1953 .

[2]  Lila L. Gatlin,et al.  Information theory and the living system , 1972 .

[3]  Werner Ebeling,et al.  Entropy of symbolic sequences: the role of correlations , 1991 .

[4]  Werner Ebeling,et al.  Word frequency and entropy of symbolic sequences: a dynamical perspective , 1992 .

[5]  W. Ebeling,et al.  A New Method to Calculate Higher-Order Entropies from Finite Samples , 1993 .

[6]  Werner Ebeling,et al.  Physik der Evolutionsprozesse , 1990 .

[7]  Wentian Li,et al.  Mutual Information Functions of Natural Language Texts , 1989 .

[8]  R. Rainger,et al.  The dynamics of evolution , 1995 .

[9]  Werner Ebeling,et al.  Dynamics and Complexity of Biomolecules , 1987 .

[10]  R. Voss,et al.  Evolution of long-range fractal correlations and 1/f noise in DNA base sequences. , 1992, Physical review letters.

[11]  C. Peng,et al.  Long-range correlations in nucleotide sequences , 1992, Nature.

[12]  W. Ebeling,et al.  Power law distributions of spectral density and higher order entropies , 1994 .

[13]  H. Herzel.  Complexity of symbol sequences , 1988 .

[14]  P. Grassberger.  Finite sample corrections to entropy and dimension estimates , 1988 .

[15]  Matts Roos,et al.  MINUIT-a system for function minimization and analysis of the parameter errors and correlations , 1984 .

[16]  Alstrom,et al.  Self-organized criticality in the "game of Life" , 1994, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[17]  Jun Zhang,et al.  LONG RANGE CORRELATION IN HUMAN WRITINGS , 1993 .

[18]  Christopher G. Langton,et al.  Computation at the edge of chaos: Phase transitions and emergent computation , 1990 .

[19]  W. Ebeling.  Chaos, order, and information in the evolution of strings , 1993 .

[20]  W. Hilberg,et al.  Der bekannte Grenzwert der redundanzfreien Information in Texten - eine Fehlinterpretation der Shannonschen Experimente? , 1990 .

[21]  J. Nicolis,et al.  Chaos and information processing , 1991 .

[22]  W. Ebeling,et al.  Finite sample effects in sequence analysis , 1994 .

[23]  Lev B. Levitin,et al.  Entropy of natural languages: Theory and experiment , 1994 .

[24]  W. Ebeling,et al.  Guessing probability distributions from small samples , 1995, cond-mat/0203467.