Transfer Entropy in Neuroscience

Information transfer is a key component of information processing, next to information storage and modification. Information transfer can be measured by a variety of directed informationmeasures of which transfer entropy is themost popular, andmost principled one. This chapter presents the basic concepts behind transfer entropy in an intuitive fashion, including graphical depictions of the key concepts. It also includes a special section devoted to the correct interpretation of the measure, especially with respect to concepts of causality. The chapter also provides an overview of estimation techniques for transfer entropy and pointers to popular open source toolboxes. It also introduces recent extensions of transfer entropy that serve to estimate delays involved in information transfer in a network. By touching upon alternative measures of information transfer, such as Massey’s directed information transfer and Runge’s momentary information transfer, it may serve as a frame of reference for more specialised treatments and as an overview over the field of studies in information transfer in general.

[1]  S. Pethel,et al.  Distinguishing anticipation from causality: anticipatory bias in the estimation of information flow. , 2011, Physical review letters.

[2]  Jakob Heinzle,et al.  Multivariate information-theoretic measures reveal directed information structure and task relevant changes in fMRI connectivity , 2010, Journal of Computational Neuroscience.

[3]  Mikhail Prokopenko,et al.  Differentiating information transfer and causal effect , 2008, 0812.4373.

[4]  J. Victor.  Binless strategies for estimation of information from neural data. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[5]  Boris Gourévitch,et al.  Evaluating information transfer between auditory cortical neurons. , 2007, Journal of neurophysiology.

[6]  A. Lapidoth,et al.  On the entropy of the sum and of the difference of independent random variables , 2008, 2008 IEEE 25th Convention of Electrical and Electronics Engineers in Israel.

[7]  Xiaoli Li,et al.  Estimating coupling direction between neuronal populations with permutation conditional mutual information , 2010, NeuroImage.

[8]  Gordon Pipa,et al.  Assessing coupling dynamics from an ensemble of time series , 2010, Entropy.

[9]  Natasa Kovacevic,et al.  Exploring transient transfer entropy based on a group-wise ICA decomposition of EEG data , 2010, NeuroImage.

[10]  Joseph T. Lizier,et al.  Multivariate construction of effective computational networks from observational data , 2012 .

[11]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[12]  Mikhail Prokopenko,et al.  Guided Self-Organization: Inception , 2014 .

[13]  Albert Y. Zomaya,et al.  Local information transfer as a spatiotemporal filter for complex systems. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[14]  Bernhard Schölkopf,et al.  Causal relationships between frequency bands of extracellular signals in visual cortex revealed by an information theoretic analysis , 2010, Journal of Computational Neuroscience.

[15]  Viola Priesemann,et al.  TRENTOOL: A Matlab open source toolbox to analyse information flow in time series data with transfer entropy , 2011, BMC Neuroscience.

[16]  Sungroh Yoon,et al.  Entropy-Based Analysis and Bioinformatics-Inspired Integration of Global Economic Information Transfer , 2013, PloS one.

[17]  Gordon Pipa,et al.  Transfer entropy—a model-free measure of effective connectivity for the neurosciences , 2010, Journal of Computational Neuroscience.

[18]  Shawn D. Pethel,et al.  Transfer Entropy for Coupled Autoregressive Processes , 2013, Entropy.

[19]  Theo Geisel,et al.  Model-Free Reconstruction of Excitatory Neuronal Connectivity from Calcium Imaging Signals , 2012, PLoS Comput. Biol..

[20]  G. Barnes,et al.  Assessing interactions of linear and nonlinear neuronal sources using MEG beamformers: a proof of concept , 2005, Clinical Neurophysiology.

[21]  Herbert Witte,et al.  Development of interaction measures based on adaptive non-linear time series analysis of biomedical signals / Entwicklung von Interaktionsmaßen auf der Grundlage adaptiver, nichtlinearer Zeitreihenanalyse von biomedizinischen Signalen , 2006, Biomedizinische Technik. Biomedical engineering.

[22]  Sergio Martinoia,et al.  Evaluation of the Performance of Information Theory-Based Methods and Cross-Correlation to Estimate the Functional Connectivity in Cortical Networks , 2009, PloS one.

[23]  Luca Faes,et al.  Bivariate nonlinear prediction to quantify the strength of complex dynamical interactions in short-term cardiovascular variability , 2006, Medical and Biological Engineering and Computing.

[24]  Matthäus Staniek,et al.  Symbolic transfer entropy: inferring directionality in biosignals , 2009, Biomedizinische Technik. Biomedical engineering.

[25]  Schreiber,et al.  Measuring information transfer , 2000, Physical review letters.

[26]  Daniel Polani,et al.  Information Flows in Causal Networks , 2008, Adv. Complex Syst..

[27]  Mikhail Prokopenko,et al.  Information Dynamics in Small-World Boolean Networks , 2011, Artificial Life.

[28]  Ulrich Meyer,et al.  Revisiting Wiener's principle of causality — interaction-delay reconstruction using transfer entropy and multivariate analysis on delay-weighted graphs , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[29]  Christoph Salge,et al.  A Bivariate Measure of Redundant Information , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[30]  Albert Y. Zomaya,et al.  The local information dynamics of distributed computation in complex systems , 2012 .

[31]  Anthony Randal McIntosh,et al.  Empirical and Theoretical Aspects of Generation and Transfer of Information in a Neuromagnetic Source Network , 2011, Front. Syst. Neurosci..

[32]  Kenneth J. Smith,et al.  Conduction in Segmentally Demyelinated Mammalian Central Axons , 1997, The Journal of Neuroscience.

[33]  H. Marko,et al.  The Bidirectional Communication Theory - A Generalization of Information Theory , 1973, IEEE Transactions on Communications.

[34]  Jochen Kaiser,et al.  Transfer entropy in magnetoencephalographic data: quantifying information flow in cortical and cerebellar networks. , 2011, Progress in biophysics and molecular biology.

[35]  B. Pompe,et al.  Momentary information transfer as a coupling measure of time series. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[36]  Albert Y. Zomaya,et al.  Information modification and particle collisions in distributed computation. , 2010, Chaos.

[37]  William W. Lytton,et al.  Synaptic information transfer in computer models of neocortical columns , 2011, Journal of Computational Neuroscience.

[38]  M. A. A. Barbosa,et al.  Entropy reduction effect imposed by hydrogen bond formation on protein folding cooperativity: evidence from a hydrophobic minimalist model. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[39]  Nikos K Logothetis,et al.  Testing methodologies for the nonlinear analysis of causal relationships in neurovascular coupling. , 2010, Magnetic resonance imaging.

[40]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[41]  Luca Faes,et al.  Compensated Transfer Entropy as a Tool for Reliably Estimating Information Transfer in Physiological Time Series , 2013, Entropy.

[42]  F. Attneave,et al.  The Organization of Behavior: A Neuropsychological Theory , 1949 .

[43]  W. Singer,et al.  Impaired Gamma-Band Activity during Perceptual Organization in Adults with Autism Spectrum Disorders: Evidence for Dysfunctional Network Activity in Frontal-Posterior Cortices , 2012, The Journal of Neuroscience.

[44]  M Palus,et al.  Synchronization as adjustment of information rates: detection from bivariate time series. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[45]  E. Beckenbach,et al.  Modern mathematics for the engineer , 1958 .

[46]  Sonja Grün,et al.  How local is the local field potential? , 2011, BMC Neuroscience.

[47]  Parlitz,et al.  Fast nearest-neighbor searching for nonlinear signal processing , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[48]  Annette Witt,et al.  Dynamic Effective Connectivity of Inter-Areal Brain Circuits , 2011, PLoS Comput. Biol..

[49]  Chi K. Tse,et al.  Optimal embedding parameters: a modelling paradigm , 2004 .

[50]  John M. Beggs,et al.  Extending Transfer Entropy Improves Identification of Effective Connectivity in a Spiking Cortical Network Model , 2011, PloS one.

[51]  Okyu Kwon,et al.  Information flow between stock indices , 2008, 0802.1747.

[52]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[53]  Vasily A. Vakorin,et al.  Confounding effects of indirect connections on causality estimation , 2009, Journal of Neuroscience Methods.

[54]  F. Atay,et al.  Information storage, loop motifs, and clustered structure in complex networks. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[55]  J. Martinerie,et al.  Statistical assessment of nonlinear causality: application to epileptic EEG signals , 2003, Journal of Neuroscience Methods.

[56]  Luca Faes,et al.  Non-uniform multivariate embedding to assess the information transfer in cardiovascular and cardiorespiratory variability series , 2012, Comput. Biol. Medicine.

[57]  K. Müller,et al.  Robustly estimating the flow direction of information in complex physical systems. , 2007, Physical review letters.

[58]  K. Tsakalis,et al.  Information Flow and Application to Epileptogenic Focus Localization From Intracranial EEG , 2009, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[59]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[60]  Gustavo Deco,et al.  Optimal Information Transfer in the Cortex through Synchronization , 2010, PLoS Comput. Biol..

[61]  T. Schreiber,et al.  Information transfer in continuous processes , 2002 .

[62]  J. Massey.  CAUSALITY, FEEDBACK AND DIRECTED INFORMATION , 1990 .

[63]  L. Faes,et al.  Information-based detection of nonlinear Granger causality in multivariate processes via a nonuniform embedding technique. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[64]  Mario Ragwitz,et al.  Markov models from data by simple nonlinear time series predictors in delay embedding spaces. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[65]  Olivier J. J. Michel,et al.  The relation between Granger causality and directed information theory: a review , 2012, Entropy.

[66]  J. Ford,et al.  Schizophrenia, myelination, and delayed corollary discharges: a hypothesis. , 2012, Schizophrenia bulletin.

[67]  David Marr,et al.  VISION A Computational Investigation into the Human Representation and Processing of Visual Information , 2009 .

[68]  Joseph T. Lizier,et al.  Moving Frames of Reference, Relativity and Invariance in Transfer Entropy and Information Dynamics , 2013, Entropy.

[69]  A. Ledberg,et al.  When two become one: the limits of causality analysis of brain dynamics. , 2012, PloS one.

[70]  Christof Koch,et al.  Quantifying synergistic mutual information , 2012, ArXiv.

[71]  Randall D. Beer,et al.  Generalized Measures of Information Transfer , 2011, ArXiv.

[72]  Olivier J. J. Michel,et al.  On directed information theory and Granger causality graphs , 2010, Journal of Computational Neuroscience.

[73]  W. Singer,et al.  Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties , 1989, Nature.

[74]  A. Seth,et al.  Granger causality and transfer entropy are equivalent for Gaussian variables. , 2009, Physical review letters.

[75]  Joseph T. Lizier,et al.  Towards a synergy-based approach to measuring information modification , 2013, 2013 IEEE Symposium on Artificial Life (ALife).

[76]  Viola Priesemann,et al.  Measuring Information-Transfer Delays , 2013, PloS one.

[77]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[78]  Randall D. Beer,et al.  Nonnegative Decomposition of Multivariate Information , 2010, ArXiv.

[79]  W. Singer,et al.  Testing non-linearity and directedness of interactions between neural groups in the macaque inferotemporal cortex , 1999, Journal of Neuroscience Methods.

[80]  Francisco Javier Díaz Pernas,et al.  Efficient Transfer Entropy Analysis of Non-Stationary Neural Time Series , 2014, PloS one.