Causation entropy from symbolic representations of dynamical systems.

Identification of causal structures and quantification of direct information flows in complex systems is a challenging yet important task, with practical applications in many fields. Data generated by dynamical processes or large-scale systems are often symbolized, either because of the finite resolution of the measurement apparatus, or because of the need of statistical estimation. By algorithmic application of causation entropy, we investigated the effects of symbolization on important concepts such as Markov order and causal structure of the tent map. We uncovered that these quantities depend nonmonotonically and, most of all, sensitively on the choice of symbolization. Indeed, we show that Markov order and causal structure do not necessarily converge to their original analog counterparts as the resolution of the partitioning becomes finer.

[1]  Igor Vajda,et al.  Estimation of the Information by an Adaptive Partitioning of the Observation Space , 1999, IEEE Trans. Inf. Theory.

[2]  Daniele Marinazzo,et al.  Causal Information Approach to Partial Conditioning in Multivariate Data Sets , 2011, Comput. Math. Methods Medicine.

[3]  B. Pompe,et al.  Momentary information transfer as a coupling measure of time series. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[4]  B. Pompe,et al.  Permutation entropy: a natural complexity measure for time series. , 2002, Physical review letters.

[5]  J. Eisert,et al.  Extracting dynamical equations from experimental data is NP hard. , 2010, Physical review letters.

[6]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[7]  Qing Gao,et al.  Evaluation of effective connectivity of motor areas during motor imagery and execution using conditional Granger causality , 2011, NeuroImage.

[8]  Dimitris Kugiumtzis,et al.  Transfer Entropy on Rank Vectors , 2010, ArXiv.

[9]  N. Ay,et al.  Closure measures for coarse-graining of the tent map. , 2014, Chaos.

[10]  S. Frenzel,et al.  Partial mutual information for coupling analysis of multivariate time series. , 2007, Physical review letters.

[11]  Gordon Pipa,et al.  Transfer entropy—a model-free measure of effective connectivity for the neurosciences , 2010, Journal of Computational Neuroscience.

[12]  Dimitris Kugiumtzis,et al.  Non-uniform state space reconstruction and coupling detection , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[13]  M. Paluš,et al.  Inferring the directionality of coupling with conditional mutual information. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[14]  Douglas Lind,et al.  An Introduction to Symbolic Dynamics and Coding , 1995 .

[15]  Jürgen Kurths,et al.  Quantifying Causal Coupling Strength: A Lag-specific Measure For Multivariate Time Series Related To Transfer Entropy , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[16]  Ying-Cheng Lai,et al.  Communicating with chaos using two-dimensional symbolic dynamics , 1999 .

[17]  B. Kitchens.  Symbolic Dynamics: One-sided, Two-sided and Countable State Markov Shifts , 1997 .

[18]  Luca Faes,et al.  Effect of Age on Complexity and Causality of the Cardiovascular Control: Comparison between Model-Based and Model-Free Approaches , 2014, PloS one.

[19]  Franz von Kutschera,et al.  Causation , 1993, J. Philos. Log..

[20]  Mario Ragwitz,et al.  Markov models from data by simple nonlinear time series predictors in delay embedding spaces. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[21]  K. Kendrick,et al.  Partial Granger causality—Eliminating exogenous inputs and latent variables , 2008, Journal of Neuroscience Methods.

[22]  P. Grassberger,et al.  Generating partitions for the dissipative Hénon map , 1985 .

[23]  Y. Lai,et al.  What symbolic dynamics do we get with a misplaced partition? On the validity of threshold crossings analysis of chaotic time-series , 2001 .

[24]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[25]  Andrew G. Glen,et al.  APPL , 2001 .

[26]  Matthäus Staniek,et al.  Symbolic transfer entropy. , 2008, Physical review letters.

[27]  N. Packard,et al.  Symbolic dynamics of one-dimensional maps: Entropies, finite precision, and noise , 1982 .

[28]  Annick A Lesne.  Chaos in biology. , 2006, Rivista di biologia.

[29]  Hiroshi Teramoto,et al.  How does a choice of Markov partition affect the resultant symbolic dynamics? , 2010, Chaos.

[30]  Jie Sun,et al.  Identifying the Coupling Structure in Complex Systems through the Optimal Causation Entropy Principle , 2014, Entropy.

[31]  A. Boyarsky,et al.  Laws of Chaos : Invariant Measures and Dynamical Systems in One Dimension , 1997 .

[32]  Lai,et al.  Estimating generating partitions of chaotic systems by unstable periodic orbits , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[33]  A. Seth,et al.  Multivariate Granger causality and generalized variance. , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[34]  Erik M. Bollt,et al.  Causation entropy identifies indirect influences, dominance of neighbors and anticipatory couplings , 2014, 1504.03769.

[35]  K. Hlavácková-Schindler,et al.  Causality detection based on information-theoretic approaches in time series analysis , 2007 .

[36]  P. Grassberger.  Entropy Estimates from Insufficient Samplings , 2003, physics/0307138.

[37]  Dimitris Kugiumtzis,et al.  Markov Chain Order estimation with Conditional Mutual Information , 2013, ArXiv.

[38]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[39]  A. Porta,et al.  K-nearest-neighbor conditional entropy approach for the assessment of the short-term complexity of cardiovascular control , 2013, Physiological measurement.

[40]  S. D. Pethel,et al.  Exact significance test for Markov order , 2014 .

[41]  L. Kocarev,et al.  The permutation entropy rate equals the metric entropy rate for ergodic information sources and ergodic dynamical systems , 2005, nlin/0503044.

[42]  J. Yorke,et al.  Chaos: An Introduction to Dynamical Systems , 1997 .

[43]  J. Heckman,et al.  Econometric Causality , 2008 .

[44]  S. Capozziello,et al.  On Relativistic Quantum Information Properties of Entangled Wave Vectors of Massive Fermions , 2011, 1111.2348.

[45]  Erik M. Bollt,et al.  Applied and Computational Measurable Dynamics , 2013, Mathematical modeling and computation.

[46]  Lai,et al.  Validity of threshold-crossing analysis of symbolic dynamics from chaotic time series , 2000, Physical review letters.

[47]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[48]  Schreiber,et al.  Measuring information transfer , 2000, Physical review letters.

[49]  Dane Taylor,et al.  Causal Network Inference by Optimal Causation Entropy , 2014, SIAM J. Appl. Dyn. Syst..

[50]  P. Grassberger.  Finite sample corrections to entropy and dimension estimates , 1988 .

[51]  S. Greenland,et al.  Causation and causal inference in epidemiology. , 2005, American journal of public health.

[52]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[53]  J. Eckmann,et al.  Iterated maps on the interval as dynamical systems , 1980 .

[54]  Jürgen Kurths,et al.  Escaping the curse of dimensionality in estimating multivariate transfer entropy. , 2012, Physical review letters.

[55]  J. Zukas.  Introduction to the Modern Theory of Dynamical Systems , 1998 .

[56]  Richard F. Gunst,et al.  Applied Regression Analysis , 1999, Technometrics.

[57]  Kohei Nakajima,et al.  Symbolic transfer entropy rate is equal to transfer entropy rate for bivariate finite-alphabet stationary ergodic Markov processes , 2011, 1112.2493.

[58]  R. Gallager.  Stochastic Processes , 2014 .