Information dynamics : Premises, challenges and results

In various fields of contemporary research information and dynamics are becoming the key terms. Theoretic information reasoning is well known in physics, especially in thermodynamics where the relationship between the statistical (or, informational) entropy of the system and its thermodynamical entropy have been studied since a long time. Information theory is especially relevant to data processing and statistical inference. Generally speaking, the apparatus of information theory is applicable to any probabilistic system of observations since whenever we make statistical observations (or design and conduct statistical experiments) we seek information. When the language of information theory (the concepts of entropy, mutual information between random variables and processes, information rate, maximum entropy formalism, information flow, etc.) is used in connection with system dynamics we come to the notion of information dynamics. The objective of this report is to show a potential of the basic information theoretic methodology for the analysis of various problems of system dynamics. In particular, we wish to indicate some challenges and expound our recent results on the maximum information entropy approach to the analysis of stochastic dynamical systems.

[1]  Claude E. Shannon,et al.  A mathematical theory of communication , 1948, MOCO.

[2]  X. D. Hemptinne.  Non-equilibrium statistical thermodynamics , 1995 .

[3]  George J. Klir,et al.  Fuzzy sets, uncertainty and information , 1988 .

[4]  Yuri S. Kivshar,et al.  Dynamics of Solitons in Nearly Integrable Systems , 1989 .

[5]  R. Ingarden,et al.  Information Dynamics and Open Systems: Classical and Quantum Approach , 1997 .

[6]  Gregory J. Chaitin,et al.  Algorithmic Information Theory , 1987, IBM J. Res. Dev..

[7]  J. Nadal,et al.  From statistical physics to statistical inference and back , 1994 .

[8]  K. Sobczyk,et al.  Approximate probability distributions for stochastic systems : maximum entropy method , 1999 .

[9]  Sueo Sugimoto,et al.  Spectral expressions of information measures of Gaussian time series and their relation to AIC and CAT , 1988, IEEE Trans. Inf. Theory.

[10]  D. B. Preston.  Spectral Analysis and Time Series , 1983 .

[11]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[12]  Thomas M. Cover,et al.  The entropy of Markov trajectories , 1993, IEEE Trans. Inf. Theory.

[13]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[14]  A. Garrett.  Irreversibility, Probability and Entropy , 1994 .

[15]  F. Takens,et al.  On the nature of turbulence , 1971 .

[16]  Hermann Haken,et al.  Information and Self-Organization: A Macroscopic Approach to Complex Systems , 2010 .

[17]  K. Sobczyk,et al.  Maximum entropy principle and nonlinear stochastic oscillators , 1993 .

[18]  Kazimierz Sobczyk,et al.  Maximum entropy principle and non-stationary distributions of stochastic systems , 1996 .

[19]  J. Trębicki,et al.  Maximum entropy principle in stochastic dynamics , 1990 .

[20]  M. Mackey,et al.  Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics , 1998 .

[21]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[22]  Michael C. Mackey,et al.  The dynamic origin of increasing entropy , 1989 .

[23]  Amiel Feinstein,et al.  Information and information stability of random variables and processes , 1964 .

[24]  D. V. Gokhale,et al.  Kullback-Leibler information measure for studying convergence rates of densities and distributions , 1993, IEEE Trans. Inf. Theory.

[25]  R. Hartley.  Transmission of information , 1928 .

[26]  Raphael Aronson,et al.  Theory and application of the Boltzmann equation , 1976 .

[27]  R. Johnson,et al.  Properties of cross-entropy minimization , 1981, IEEE Trans. Inf. Theory.

[28]  E. T. Jaynes,et al.  Papers on probability, statistics and statistical physics , 1983 .

[29]  R. Ingarden,et al.  Information without probability , 1962 .

[30]  A. Kolmogorov.  Three approaches to the quantitative definition of information , 1968 .

[31]  A. Mohammad-Djafari,et al.  Maximum Entropy and Bayesian Methods , 1993 .

[32]  Ali Mohammad-Djafari,et al.  MAXIMUM ENTROPY AND BAYESIAN APPROACH IN TOMOGRAPHIC IMAGE RECONSTRUCTION AND RESTORATION , 1989 .