Differentiating information transfer and causal effect

Abstract The concepts of information transfer and causal effect have received much recent attention, yet often the two are not appropriately distinguished and certain measures have been suggested to be suitable for both. We discuss two existing measures, transfer entropy and information flow, which can be used separately to quantify information transfer and causal information flow respectively. We apply these measures to cellular automata on a local scale in space and time, in order to explicitly contrast them and emphasize the differences between information transfer and causality. We also describe the manner in which the measures are complementary, including the conditions under which they in fact converge. We show that causal information flow is a primary tool to describe the causal structure of a system, while information transfer can then be used to describe the emergent computation on that causal structure.

[1]  Chrystopher L. Nehaniv,et al.  Tracking Information Flow through the Environment: Simple Cases of Stigmerg , 2004 .

[2]  J. Urry.  Complexity , 2006, Interpreting Art.

[3]  K. Hlavácková-Schindler,et al.  Causality detection based on information-theoretic approaches in time series analysis , 2007 .

[4]  J. Crutchfield,et al.  The attractor—basin portrait of a cellular automaton , 1992 .

[5]  Mats G. Nordahl,et al.  Continuity of Information Transport in Surjective Cellular Automata , 2007 .

[6]  X San Liang,et al.  Information flow within stochastic dynamical systems. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[7]  BMC Bioinformatics , 2005 .

[8]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[9]  C. Moore,et al.  Automatic filters for the detection of coherent structure in spatiotemporal systems. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[10]  A. Wuensche.  Classifying cellular automata automatically: finding gliders, filtering, and relating space-time patterns, attractor basins, and the Z parameter , 1999 .

[11]  Schreiber,et al.  Measuring information transfer , 2000, Physical review letters.

[12]  James P. Crutchfield,et al.  Computational mechanics of cellular automata: an example , 1997 .

[13]  Daniel Polani,et al.  Information Flows in Causal Networks , 2008, Adv. Complex Syst..

[14]  Yasuo Kuniyoshi,et al.  Comparison of nonlinear Granger causality extensions for low-dimensional systems. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  Mats G. Nordahl,et al.  Local Information in One-Dimensional Cellular Automata , 2004, ACRI.

[16]  Martin Brown,et al.  Information-theoretic sensitivity analysis: a general method for credit assignment in complex networks , 2007, Journal of The Royal Society Interface.

[17]  P. F. Verdes.  Assessing causality from multivariate time series. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[18]  Melanie Mitchell,et al.  Computation in Cellular Automata: A Selected Review , 2005, Non-standard Computation.

[19]  Olaf Sporns,et al.  Mapping Information Flow in Sensorimotor Networks , 2006, PLoS Comput. Biol..

[20]  Phil Husbands,et al.  Tracking Information Flow through the Environment: Simple Cases of Stigmergy , 2004 .

[21]  Stephen Wolfram,et al.  A New Kind of Science , 2003, Artificial Life.

[22]  Y. Yoshikawa,et al.  Causality detected by transfer entropy leads acquisition of joint attention , 2007, 2007 IEEE 6th International Conference on Development and Learning.

[23]  L Jaeger,et al.  Top-down causation by information control: from a philosophical problem to a scientific research programme , 2007, Journal of The Royal Society Interface.

[24]  C. Granger.  Investigating Causal Relations by Econometric Models and Cross-Spectral Methods , 1969 .

[25]  Doheon Lee,et al.  Inferring Gene Regulatory Networks from Microarray Time Series Data Using Transfer Entropy , 2007, Twentieth IEEE International Symposium on Computer-Based Medical Systems (CBMS'07).

[26]  Marc M. Van Hulle,et al.  Information Theoretic Derivations for Causality Detection: Application to Human Gait , 2007, ICANN.

[27]  Albert Y. Zomaya,et al.  Local information transfer as a spatiotemporal filter for complex systems. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[28]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[29]  Y. Kuniyoshi,et al.  Detecting direction of causal interactions between dynamically coupled signals. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[30]  Ursula Kummer,et al.  Information transfer in signaling pathways: A study using coupled simulated and experimental data , 2008, BMC Bioinformatics.

[31]  Ronald H. Epp.  Aristotle: A Contemporary Appreciation , 1975 .

[32]  Yao-Chen Hung,et al.  Chaotic communication via temporal transfer entropy. , 2008, Physical review letters.

[33]  M R DeWeese,et al.  How to measure the information gained from one symbol. , 1999, Network.

[34]  Yasuo Kuniyoshi,et al.  Methods for Quantifying the Causal Structure of bivariate Time Series , 2007, Int. J. Bifurc. Chaos.

[35]  Albert Y. Zomaya,et al.  A framework for the local information dynamics of distributed computation in complex systems , 2008, ArXiv.

[36]  M. Paluš,et al.  Inferring the directionality of coupling with conditional mutual information. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.