Autocorrelation versus entropy-based autoinformation for measuring dependence in random signal

This paper studies in conjunction correlation and entropy-based information measures for the characterization of statistical dependence in a random signal. Several simple reference models of random signal are presented, for which both the autocorrelation and autoinformation functions are calculated explicitly in analytical form. Conditions are investigated where a general relation is shown to exist between these two functions in asymptotic regime, and which especially apply to stationary signals. Another, recent, model of nonstationary random signal with long-range dependence, is also presented and analyzed. For this model, the autocorrelation and autoinformation functions are calculated and compared for the first time, and exhibit more complex asymptotic behavior. This paper is intended to provide essentially theoretical models and results useful for better appreciation of the potentialities of the autoinformation function, in complement to the more common autocorrelation function, for the study of structures, informational contents and properties of complex random signals.

[1]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[2]  Lei Feng,et al.  A method for segmentation of switching dynamic modes in time series , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  Generation of signals with long-range correlations , 2001 .

[4]  Sergey V. Buldyrev,et al.  Long-range power-law correlations in condensed matter physics and biophysics , 1993 .

[5]  Moon,et al.  Estimation of mutual information using kernel density estimators. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[6]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[7]  J. Beran.  Statistical methods for data with long-range dependence , 1992 .

[8]  Christoph Kayser,et al.  Temporal Correlations of Orientations in Natural Scenes , 2002, Neurocomputing.

[9]  Aarnout Brombacher,et al.  Probability... , 2009, Qual. Reliab. Eng. Int..

[10]  Wentian Li,et al.  The Study of Correlation Structures of DNA Sequences: A Critical Review , 1997, Comput. Chem..

[11]  W. Ebeling,et al.  Entropy and Long-Range Correlations in Literary English , 1993, cond-mat/0204108.

[12]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[13]  R. Voss,et al.  Evolution of long-range fractal correlations and 1/f noise in DNA base sequences. , 1992, Physical review letters.

[14]  Igor Vajda,et al.  Estimation of the Information by an Adaptive Partitioning of the Observation Space , 1999, IEEE Trans. Inf. Theory.

[15]  Ebeling,et al.  Entropies of biosequences: The role of repeats. , 1994, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[16]  I. Grosse,et al.  MEASURING CORRELATIONS IN SYMBOL SEQUENCES , 1995 .

[17]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[18]  R. Moddemeijer.  On estimation of entropy and mutual information of continuous distributions , 1989 .

[19]  Marc-Thorsten Hütt,et al.  Information theory reveals large-scale synchronisation of statistical correlations in eukaryote genomes. , 2005, Gene.

[20]  Ki H. Chon,et al.  Mutual information function assesses autonomic information flow of heart rate dynamics at different time scales , 2005, IEEE Transactions on Biomedical Engineering.

[21]  (max, +) Dynamics Systems for Modeling Traffic with Long-Range Dependence , 1998 .

[22]  A. Oppenheim,et al.  Signal processing with fractals: a wavelet-based approach , 1996 .

[23]  G. Darbellay,et al.  The entropy as a tool for analysing statistical dependences in financial time series , 2000 .

[24]  Fraser,et al.  Independent coordinates for strange attractors from mutual information. , 1986, Physical review. A, General physics.

[25]  Multifractal Fourier spectra and power-law decay of correlations in random substitution sequences. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[26]  Derek Abbott,et al.  MUTUAL INFORMATION FOR EXAMINING CORRELATIONS IN DNA , 2004 .

[27]  Wentian Li.  Mutual information functions versus correlation functions , 1990 .

[28]  Li,et al.  Expansion-modification systems: A model for spatial 1/f spectra. , 1991, Physical review. A, Atomic, molecular, and optical physics.