Localization and regularization of normalized transfer entropy

To find hidden structures of a data set, it is important to understand the relationship between variables such as genes or neurons. As a measure of such relationship, causality is to find directed relations between the variables, which can reveal more of the structures than undirected relations. As a quantitative measure of such causal relationship, transfer entropy has been proposed and successfully applied to capture the amount of information flow between events and sequences. In order to analyze the flow locally in time, we propose to localize normalized transfer entropy and regularize it to avoid the unstable result. Experiment results with synthetic and real-world data confirm the usefulness of our algorithm.

[1]  Boris Gourévitch,et al.  Evaluating information transfer between auditory cortical neurons. , 2007, Journal of neurophysiology.

[2]  Yao-Chen Hung,et al.  Chaotic communication via temporal transfer entropy. , 2008, Physical review letters.

[3]  Its Appkications,et al.  ISITA '90 : 1990 International Symposium on Information Theory and Its Applications, November 27-30, 1990, Sheraton Waikiki Hotel, Hawaii : Proceedings , 1990 .

[4]  H. Marko,et al.  The Bidirectional Communication Theory - A Generalization of Information Theory , 1973, IEEE Transactions on Communications.

[5]  Randall D. Beer,et al.  Information Dynamics of Evolved Agents , 2010, SAB.

[6]  C. Granger.  Investigating causal relations by econometric models and cross-spectral methods , 1969 .

[7]  Olaf Sporns,et al.  From Data Streams to Information Flow: Information Exchange in Child-Parent Interaction , 2011, CogSci.

[8]  J. Massey.  CAUSALITY, FEEDBACK AND DIRECTED INFORMATION , 1990 .

[9]  Heeyoul Choi,et al.  Dynamic learning for visual representation of asymmetric proximity , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[10]  H. Kantz,et al.  Analysing the information flow between financial time series , 2002 .

[11]  Eamonn J. Keogh,et al.  A symbolic representation of time series, with implications for streaming algorithms , 2003, DMKD '03.

[12]  William Bialek,et al.  Entropy and Inference, Revisited , 2001, NIPS.

[13]  H. Sebastian Seung,et al.  The Manifold Ways of Perception , 2000, Science.

[14]  Heeyoul Choi,et al.  Robust kernel Isomap , 2007, Pattern Recognit..

[15]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[16]  Albert Y. Zomaya,et al.  Local information transfer as a spatiotemporal filter for complex systems. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[17]  Schreiber,et al.  Measuring information transfer , 2000, Physical review letters.