Information Decomposition in Bivariate Systems: Theory and Application to Cardiorespiratory Dynamics

Abstract: In the framework of information dynamics, the temporal evolution of coupled systems can be studied by decomposing the predictive information about an assigned target system into amounts quantifying the information stored inside the system and the information transferred to it. While information storage and transfer are computed through the known self-entropy (SE) and transfer entropy (TE), an alternative decomposition evidences the so-called cross entropy (CE) and conditional SE (cSE), quantifying the cross information and internal information of the target system, respectively. This study presents a thorough evaluation of SE, TE, CE and cSE as quantities related to the causal statistical structure of coupled dynamic processes. First, we investigate the theoretical properties of these measures, providing the conditions for their existence and assessing the meaning of the information theoretic quantity that each of them reflects. Then, we present an approach for the exact computation of information dynamics based on the linear Gaussian approximation, and exploit this approach to characterize the behavior of SE, TE, CE and cSE in benchmark systems with known dynamics. Finally, we exploit these measures to study cardiorespiratory dynamics measured from healthy subjects during head-up tilt and paced breathing protocols. Our main result is that the combined evaluation of the measures of information dynamics allows to infer the causal effects associated with the observed

[1]  Luca Faes,et al.  Conditional Entropy-Based Evaluation of Information Dynamics in Physiological Systems , 2014 .

[2]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  C. W. J. Granger,et al.  Economic Processes Involving Feedback , 1963, Inf. Control..

[4]  Gordon Pipa,et al.  Transfer entropy—a model-free measure of effective connectivity for the neurosciences , 2010, Journal of Computational Neuroscience.

[5]  Luca Faes,et al.  Lag-Specific Transfer Entropy as a Tool to Assess Cardiovascular and Cardiorespiratory Information Transfer , 2014, IEEE Transactions on Biomedical Engineering.

[6]  C. Granger.  Testing for causality: a personal viewpoint , 1980 .

[7]  P. V. E. McClintock,et al.  Evolution of cardiorespiratory interactions with age , 2013, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[8]  Adam B. Barrett,et al.  Granger causality is designed to measure effect, not mechanism , 2013, Front. Neuroinform..

[9]  A. Seth,et al.  Multivariate Granger causality and generalized variance. , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[10]  A. Malliani,et al.  Heart rate variability. Standards of measurement, physiological interpretation, and clinical use , 1996 .

[11]  Mikhail Prokopenko,et al.  Differentiating information transfer and causal effect , 2008, 0812.4373.

[12]  J. Taylor,et al.  Short‐term cardiovascular oscillations in man: measuring and modelling the physiologies , 2002, The Journal of physiology.

[13]  Mohamed Najim,et al.  Consistent estimation of autoregressive parameters from noisy observations based on two interacting Kalman filters , 2006, Signal Process..

[14]  T. Schreiber,et al.  Information transfer in continuous processes , 2002 .

[15]  Luca Faes,et al.  Non-uniform multivariate embedding to assess the information transfer in cardiovascular and cardiorespiratory variability series , 2012, Comput. Biol. Medicine.

[16]  D. Eckberg,et al.  Time-frequency methods and voluntary ramped-frequency breathing: a powerful combination for exploration of human neurophysiological mechanisms. , 2013, Journal of applied physiology.

[17]  Albert Y. Zomaya,et al.  Local measures of information storage in complex distributed computation , 2012, Inf. Sci..

[18]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[19]  M. Rosenblum,et al.  In vivo cardiac phase response curve elucidates human respiratory heart rate variability , 2013, Nature Communications.

[20]  Francisco Javier Díaz Pernas,et al.  Efficient Transfer Entropy Analysis of Non-Stationary Neural Time Series , 2014, PloS one.

[21]  Luca Faes,et al.  Measuring Connectivity in Linear Multivariate Processes: Definitions, Interpretation, and Practical Analysis , 2012, Comput. Math. Methods Medicine.

[22]  A. Ledberg,et al.  Framework to study dynamic dependencies in networks of interacting processes. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[23]  A. Penn,et al.  Reciprocal regulation of A-to-I RNA editing and the vertebrate nervous system , 2013, Front. Neurosci..

[24]  Arun K. Tangirala,et al.  Quantitative analysis of directional strengths in jointly stationary linear multivariate processes , 2010, Biological Cybernetics.

[25]  Jürgen Kurths,et al.  Escaping the curse of dimensionality in estimating multivariate transfer entropy. , 2012, Physical review letters.

[26]  A. Ledberg,et al.  When two become one: the limits of causality analysis of brain dynamics. , 2012, PloS one.

[27]  Jürgen Kurths,et al.  Quantifying Causal Coupling Strength: A Lag-specific Measure For Multivariate Time Series Related To Transfer Entropy , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[28]  A. Malliani,et al.  Information domain analysis of cardiovascular variability signals: Evaluation of regularity, synchronisation and co-ordination , 2000, Medical and Biological Engineering and Computing.

[29]  J. Cacioppo,et al.  Respiratory sinus arrhythmia: autonomic origins, physiological mechanisms, and psychophysiological implications. , 1993, Psychophysiology.

[30]  Albert Y. Zomaya,et al.  The local information dynamics of distributed computation in complex systems , 2012 .

[31]  Karl J. Friston,et al.  Dynamic causal modelling , 2003, NeuroImage.

[32]  Viola Priesemann,et al.  Local active information storage as a tool to understand distributed neural information processing , 2013, Front. Neuroinform..

[33]  Karl J. Friston,et al.  Effective connectivity: Influence, causality and biophysical modeling , 2011, NeuroImage.

[34]  Luca Faes,et al.  Information decomposition of short-term cardiovascular and cardiorespiratory variability , 2013, Computing in Cardiology 2013.

[35]  Dimitris Kugiumtzis,et al.  Direct coupling information measure from non-uniform embedding , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[36]  Anil K. Seth,et al.  The MVGC multivariate Granger causality toolbox: A new approach to Granger-causal inference , 2014, Journal of Neuroscience Methods.

[37]  L. Faes,et al.  Information Domain Approach to the Investigation of Cardio-Vascular, Cardio-Pulmonary, and Vasculo-Pulmonary Causal Couplings , 2011, Front. Physio..

[38]  Sabine Van Huffel,et al.  Investigating cardiac and respiratory determinants of heart rate variability in an information-theoretic framework , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[39]  C. Braun,et al.  Adaptive AR modeling of nonstationary time series by means of Kalman filtering , 1998, IEEE Transactions on Biomedical Engineering.

[40]  A. Porta,et al.  Model-based assessment of baroreflex and cardiopulmonary couplings during graded head-up tilt , 2012, Comput. Biol. Medicine.

[41]  Luca Faes,et al.  Effect of Age on Complexity and Causality of the Cardiovascular Control: Comparison between Model-Based and Model-Free Approaches , 2014, PloS one.

[42]  L. Faes,et al.  Information-based detection of nonlinear Granger causality in multivariate processes via a nonuniform embedding technique. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[43]  L. Faes,et al.  Information dynamics of brain–heart physiological networks during sleep , 2014, New Journal of Physics.

[44]  A. Porta,et al.  Accounting for Respiration is Necessary to Reliably Infer Granger Causality From Cardiovascular Variability Series , 2012, IEEE Transactions on Biomedical Engineering.

[45]  Mikhail Prokopenko,et al.  Information Dynamics in Small-World Boolean Networks , 2011, Artificial Life.

[46]  Markus P. Schlaich,et al.  Change in Sympathetic Nerve Firing Pattern Associated with Dietary Weight Loss in the Metabolic Syndrome , 2011, Front. Physio..

[47]  Raul Vicente,et al.  Transfer Entropy in Neuroscience , 2014 .

[48]  Dimitris Kugiumtzis,et al.  Non-uniform state space reconstruction and coupling detection , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[49]  R. Dahlhaus.  Graphical interaction models for multivariate time series1 , 2000 .

[50]  A. Malliani,et al.  Model for the assessment of heart period and arterial pressure variability interactions and of respiration influences , 1994, Medical and Biological Engineering and Computing.

[51]  Sergio Cerutti,et al.  Entropy, entropy rate, and pattern classification as tools to typify complexity in short heart period variability series , 2001, IEEE Transactions on Biomedical Engineering.

[52]  Jacques Olivier Fortrat,et al.  Respiratory influences on non-linear dynamics of heart rate variability in humans , 1997, Biological Cybernetics.

[53]  L. Faes,et al.  Investigating the mechanisms of cardiovascular and cerebrovascular regulation in orthostatic syncope through an information decomposition strategy , 2013, Autonomic Neuroscience.

[54]  Luca Faes,et al.  Estimating the decomposition of predictive information in multivariate systems. , 2015, Physical review. E, Statistical, nonlinear, and soft matter physics.

[55]  Elke Vlemincx,et al.  The effect of instructed ventilatory patterns on physiological and psychological dimensions of relaxation , 2010 .

[56]  Luca Faes,et al.  Compensated Transfer Entropy as a Tool for Reliably Estimating Information Transfer in Physiological Time Series , 2013, Entropy.

[57]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[58]  Aneta Stefanovska,et al.  Inference of time-evolving coupled dynamical systems in the presence of noise. , 2012, Physical review letters.

[59]  Schreiber,et al.  Measuring information transfer , 2000, Physical review letters.

[60]  Luca Faes,et al.  Assessing causality in brain dynamics and cardiovascular control , 2013, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[61]  A. Seth,et al.  Granger causality and transfer entropy are equivalent for Gaussian variables. , 2009, Physical review letters.

[62]  Daniel Chicharro,et al.  Algorithms of causal inference for the analysis of effective connectivity among brain regions , 2014, Front. Neuroinform..