Temporal information partitioning: Characterizing synergy, uniqueness, and redundancy in interacting environmental variables

Information theoretic measures can be used to identify non-linear interactions between source and target variables through reductions in uncertainty. In information partitioning, multivariate mutual information is decomposed into synergistic, unique, and redundant components. Synergy is information shared only when sources influence a target together, uniqueness is information only provided by one source, and redundancy is overlapping shared information from multiple sources. While this partitioning has been applied to provide insights into complex dependencies, several proposed partitioning methods overestimate redundant information and omit a component of unique information because they do not account for source dependencies. Additionally, information partitioning has only been applied to time-series data in a limited context, using basic pdf estimation techniques or a Gaussian assumption. We develop a Rescaled Redundancy measure (Rs) to solve the source dependency issue, and present Gaussian, autoregressive, and chaotic test cases to demonstrate its advantages over existing techniques in the presence of noise, various source correlations, and different types of interactions. This study constitutes the first rigorous application of information partitioning to environmental time-series data, and addresses how noise, pdf estimation technique, or source dependencies can influence detected measures. We illustrate how our techniques can unravel the complex nature of forcing and feedback within an ecohydrologic system with an application to 1-minute environmental signals of air temperature, relative humidity, and windspeed. The methods presented here are applicable to the study of a broad range of complex systems composed of interacting variables.

[1]  Praveen Kumar,et al.  Ecohydrologic process networks: 2. Analysis and characterization , 2009 .

[2]  Guorong Wu,et al.  Expanding the transfer entropy to identify information subgraphs in complex systems , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[3]  Praveen Kumar,et al.  Information Driven Ecohydrologic Self-Organization , 2010, Entropy.

[4]  Jürgen Kurths,et al.  Statistical Mechanics and Information-Theoretic Perspectives on Complexity in the Earth System , 2013, Entropy.

[5]  Paola Passalacqua,et al.  Process connectivity in a naturally prograding river delta , 2017 .

[6]  Eckehard Olbrich,et al.  Information Decomposition and Synergy , 2015, Entropy.

[7]  Daniele Marinazzo,et al.  Identification of redundant and synergetic circuits in triplets of electrophysiological data , 2014, Journal of neural engineering.

[8]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[9]  Grey Nearing,et al.  Estimating information entropy for hydrological data: One‐dimensional case , 2014 .

[10]  Marian Verhelst,et al.  Understanding Interdependency Through Complex Information Sharing , 2015, Entropy.

[11]  Atul Malhotra,et al.  Transfer Entropy Estimation and Directional Coupling Change Detection in Biomedical Time Series , 2012, Biomedical engineering online.

[12]  Jakob Runge,et al.  Quantifying information transfer and mediation along causal pathways in complex systems. , 2015, Physical review. E, Statistical, nonlinear, and soft matter physics.

[13]  Tracey Ho,et al.  Quantifying Redundant Information in Predicting a Target Random Variable , 2014, Entropy.

[14]  Adam B. Barrett,et al.  An exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  Sirish L. Shah,et al.  Direct Causality Detection via the Transfer Entropy Approach , 2013, IEEE Transactions on Control Systems Technology.

[16]  Ashish Sharma,et al.  An information theoretic alternative to model a natural system using observational information alone , 2014 .

[17]  Efi Foufoula-Georgiou,et al.  The change of nature and the nature of change in agricultural landscapes: Hydrologic regime shifts modulate ecological transitions , 2015 .

[18]  Christof Koch,et al.  Quantifying synergistic mutual information , 2012, ArXiv.

[19]  Schreiber,et al.  Measuring information transfer , 2000, Physical review letters.

[20]  Amilcare Porporato,et al.  Stochastic soil water balance under seasonal climates , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[21]  Praveen Kumar,et al.  Ecohydrologic process networks: 1. Identification , 2009 .

[22]  Benjamin Flecker,et al.  Synergy, redundancy, and multivariate information measures: an experimentalist’s perspective , 2014, Journal of Computational Neuroscience.

[23]  Christoph Salge,et al.  A Bivariate Measure of Redundant Information , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[24]  Pushpa N. Rathie,et al.  On the entropy of continuous probability distributions (Corresp.) , 1978, IEEE Trans. Inf. Theory.

[25]  Praveen Kumar,et al.  Information Theoretic Measures to Infer Feedback Dynamics in Coupled Logistic Networks , 2015, Entropy.

[26]  Anthony Longjas,et al.  Coupling freshwater mussel ecology and river dynamics using a simplified dynamic interaction model , 2016, Freshwater Science.

[27]  Praveen Kumar,et al.  Temporal Information Partitioning Networks (TIPNets): A process network approach to infer ecohydrologic shifts , 2017 .

[28]  Eckehard Olbrich,et al.  Quantifying unique information , 2013, Entropy.