Measuring Information-Transfer Delays

In complex networks such as gene networks, traffic systems or brain circuits it is important to understand how long it takes for the different parts of the network to effectively influence one another. In the brain, for example, axonal delays between brain areas can amount to several tens of milliseconds, adding an intrinsic component to any timing-based processing of information. Inferring neural interaction delays is thus needed to interpret the information transfer revealed by any analysis of directed interactions across brain structures. However, a robust estimation of interaction delays from neural activity faces several challenges if modeling assumptions on interaction mechanisms are wrong or cannot be made. Here, we propose a robust estimator for neuronal interaction delays rooted in an information-theoretic framework, which allows a model-free exploration of interactions. In particular, we extend transfer entropy to account for delayed source-target interactions, while crucially retaining the conditioning on the embedded target state at the immediately previous time step. We prove that this particular extension is indeed guaranteed to identify interaction delays between two coupled systems and is the only relevant option in keeping with Wiener’s principle of causality. We demonstrate the performance of our approach in detecting interaction delays on finite data by numerical simulations of stochastic and deterministic processes, as well as on local field potential recordings. We also show the ability of the extended transfer entropy to detect the presence of multiple delays, as well as feedback loops. While evaluated on neuroscience data, we expect the estimator to be useful in other fields dealing with network dynamics.

[1]  J. Ford,et al.  Schizophrenia, myelination, and delayed corollary discharges: a hypothesis. , 2012, Schizophrenia bulletin.

[2]  J. Kaiser,et al.  Decomposition of working memory-related scalp ERPs: crossvalidation of fMRI-constrained source analysis and ICA. , 2008, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[3]  S. Pethel,et al.  Distinguishing anticipation from causality: anticipatory bias in the estimation of information flow. , 2011, Physical review letters.

[4]  Jakob Heinzle,et al.  Multivariate information-theoretic measures reveal directed information structure and task relevant changes in fMRI connectivity , 2010, Journal of Computational Neuroscience.

[5]  Gordon Pipa,et al.  Assessing coupling dynamics from an ensemble of time series , 2010, Entropy.

[6]  A. Ledberg,et al.  When two become one: the limits of causality analysis of brain dynamics. , 2012, PloS one.

[7]  Daniel Polani,et al.  Information Flows in Causal Networks , 2008, Adv. Complex Syst..

[8]  R. Marimont,et al.  Nearest Neighbour Searches and the Curse of Dimensionality , 1979 .

[9]  Olivier J. J. Michel,et al.  On directed information theory and Granger causality graphs , 2010, Journal of Computational Neuroscience.

[10]  M. Ariel,et al.  Visual-response properties of neurons in turtle basal optic nucleus in vitro. , 1990, Journal of neurophysiology.

[11]  Mikhail Prokopenko,et al.  Differentiating information transfer and causal effect , 2008, 0812.4373.

[12]  Matthäus Staniek,et al.  Symbolic transfer entropy: inferring directionality in biosignals , 2009, Biomedizinische Technik. Biomedical engineering.

[13]  M. A. A. Barbosa,et al.  Entropy reduction effect imposed by hydrogen bond formation on protein folding cooperativity: evidence from a hydrophobic minimalist model. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[14]  Boris Gourévitch,et al.  Evaluating information transfer between auditory cortical neurons. , 2007, Journal of neurophysiology.

[15]  Michael D. Todd,et al.  Dynamic system change detection using a modification of the transfer entropy , 2009 .

[16]  Albert Y. Zomaya,et al.  Local measures of information storage in complex distributed computation , 2012, Inf. Sci..

[17]  Luca Faes,et al.  Non-uniform multivariate embedding to assess the information transfer in cardiovascular and cardiorespiratory variability series , 2012, Comput. Biol. Medicine.

[18]  K. Müller,et al.  Robustly estimating the flow direction of information in complex physical systems. , 2007, Physical review letters.

[19]  J M Nichols,et al.  Detecting nonlinearity in structural systems using the transfer entropy. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[20]  K. Tsakalis,et al.  Information Flow and Application to Epileptogenic Focus Localization From Intracranial EEG , 2009, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[21]  Sergio Martinoia,et al.  Evaluation of the Performance of Information Theory-Based Methods and Cross-Correlation to Estimate the Functional Connectivity in Cortical Networks , 2009, PloS one.

[22]  Luca Faes,et al.  Bivariate nonlinear prediction to quantify the strength of complex dynamical interactions in short-term cardiovascular variability , 2006, Medical and Biological Engineering and Computing.

[23]  Jochen Kaiser,et al.  Transfer entropy in magnetoencephalographic data: quantifying information flow in cortical and cerebellar networks. , 2011, Progress in biophysics and molecular biology.

[24]  B. Pompe,et al.  Momentary information transfer as a coupling measure of time series. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[25]  Albert Y. Zomaya,et al.  Information modification and particle collisions in distributed computation. , 2010, Chaos.

[26]  Anthony Randal McIntosh,et al.  Empirical and Theoretical Aspects of Generation and Transfer of Information in a Neuromagnetic Source Network , 2011, Front. Syst. Neurosci..

[27]  Wolf Singer,et al.  Quantifying additive evoked contributions to the event-related potential , 2012, NeuroImage.

[28]  G. Barnes,et al.  Assessing interactions of linear and nonlinear neuronal sources using MEG beamformers: a proof of concept , 2005, Clinical Neurophysiology.

[29]  Bernhard Schölkopf,et al.  Causal relationships between frequency bands of extracellular signals in visual cortex revealed by an information theoretic analysis , 2010, Journal of Computational Neuroscience.

[30]  H. Kantz,et al.  Nonlinear time series analysis , 1997 .

[31]  Joseph T. Lizier,et al.  JIDT: An Information-Theoretic Toolkit for Studying the Dynamics of Complex Systems , 2014, Front. Robot. AI.

[32]  Vasily A. Vakorin,et al.  Confounding effects of indirect connections on causality estimation , 2009, Journal of Neuroscience Methods.

[33]  Kenneth J. Smith,et al.  Conduction in Segmentally Demyelinated Mammalian Central Axons , 1997, The Journal of Neuroscience.

[34]  M Palus,et al.  Synchronization as adjustment of information rates: detection from bivariate time series. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[35]  J. Pearl Causality: Models, Reasoning and Inference , 2000 .

[36]  Viola Priesemann,et al.  TRENTOOL: A Matlab open source toolbox to analyse information flow in time series data with transfer entropy , 2011, BMC Neuroscience.

[37]  Ingo Fischer,et al.  Synchronization in simple network motifs with negligible correlation and mutual information measures. , 2012, Physical review letters.

[38]  A. Kraskov,et al.  Erratum: Estimating mutual information [Phys. Rev. E 69, 066138 (2004)] , 2011 .

[39]  Joseph T. Lizier,et al.  Multivariate construction of effective computational networks from observational data , 2012 .

[40]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[41]  Albert Y. Zomaya,et al.  Local information transfer as a spatiotemporal filter for complex systems. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[42]  Gordon Pipa,et al.  Transfer entropy—a model-free measure of effective connectivity for the neurosciences , 2010, Journal of Computational Neuroscience.

[43]  Herbert Witte,et al.  Development of interaction measures based on adaptive non-linear time series analysis of biomedical signals / Entwicklung von Interaktionsmaßen auf der Grundlage adaptiver, nichtlinearer Zeitreihenanalyse von biomedizinischen Signalen , 2006, Biomedizinische Technik. Biomedical engineering.

[44]  L. Horváth,et al.  Limit Theorems in Change-Point Analysis , 1997 .

[45]  A. N. Sharkovskiĭ Dynamic systems and turbulence , 1989 .

[46]  Michael Breakspear,et al.  An improved algorithm for the detection of dynamical interdependence in bivariate time-series , 2003, Biological Cybernetics.

[47]  Gustavo Deco,et al.  Optimal Information Transfer in the Cortex through Synchronization , 2010, PLoS Comput. Biol..

[48]  W. Singer,et al.  Testing non-linearity and directedness of interactions between neural groups in the macaque inferotemporal cortex , 1999, Journal of Neuroscience Methods.

[49]  L. Faes,et al.  Information-based detection of nonlinear Granger causality in multivariate processes via a nonuniform embedding technique. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[50]  Mario Ragwitz,et al.  Markov models from data by simple nonlinear time series predictors in delay embedding spaces. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[51]  J. Victor Binless strategies for estimation of information from neural data. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[52]  Natasa Kovacevic,et al.  Exploring transient transfer entropy based on a group-wise ICA decomposition of EEG data , 2010, NeuroImage.

[53]  J. Martinerie,et al.  Statistical assessment of nonlinear causality: application to epileptic EEG signals , 2003, Journal of Neuroscience Methods.

[54]  E M Glaser,et al.  Autapses in neocortex cerebri: synapses between a pyramidal cell's axon and its own dendrites. , 1972, Brain research.

[55]  William W. Lytton,et al.  Synaptic information transfer in computer models of neocortical columns , 2011, Journal of Computational Neuroscience.

[56]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[57]  John M. Beggs,et al.  Extending Transfer Entropy Improves Identification of Effective Connectivity in a Spiking Cortical Network Model , 2011, PloS one.

[58]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[59]  Schreiber,et al.  Measuring information transfer , 2000, Physical review letters.

[60]  Nikos K Logothetis,et al.  Testing methodologies for the nonlinear analysis of causal relationships in neurovascular coupling. , 2010, Magnetic resonance imaging.

[61]  W. Singer,et al.  Impaired Gamma-Band Activity during Perceptual Organization in Adults with Autism Spectrum Disorders: Evidence for Dysfunctional Network Activity in Frontal-Posterior Cortices , 2012, The Journal of Neuroscience.

[62]  A. Seth,et al.  Granger causality and transfer entropy are equivalent for Gaussian variables. , 2009, Physical review letters.

[63]  Voss,et al.  Anticipating chaotic synchronization , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[64]  Thomas E. Nichols,et al.  Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate , 2002, NeuroImage.

[65]  Michèle Basseville,et al.  Detection of abrupt changes: theory and application , 1993 .