Directed Information Graphs: A generalization of Linear Dynamical Graphs

We study the relationship between Directed Information Graphs (DIG) and Linear Dynamical Graphs (LDG), both of which are graphical models where nodes represent scalar random processes. DIGs are based on directed information and represent the causal dynamics between processes in a stochastic system. LDGs capture causal dynamics but only in linear dynamical systems and there are Wiener filtering to do so in a subset of LDGs. This study shows that the DIGs are generalized version of the LDGs and any strictly causal LDGs can be reconstructed through learning the corresponding DIGs.

[1]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[2]  P. A. Blight The Analysis of Time Series: An Introduction , 1991 .

[3]  Gerhard Kramer,et al.  Directed information for channels with feedback , 1998 .

[4]  D. Botstein,et al.  Cluster analysis and display of genome-wide expression patterns. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[5]  V. Latora,et al.  Complex networks: Structure and dynamics , 2006 .

[6]  Marc Timme,et al.  Revealing network connectivity from response dynamics. , 2006, Physical review letters.

[7]  Michael J. Naylor,et al.  Topology of foreign exchange markets using hierarchical structure methods , 2006, physics/0608084.

[8]  Eric D. Kolaczyk,et al.  Statistical Analysis of Network Data , 2009 .

[9]  Eric D. Kolaczyk,et al.  Statistical Analysis of Network Data: Methods and Models , 2009 .

[10]  Natasa Milic-Frayling,et al.  Statistical models of music-listening sessions in social media , 2010, WWW '10.

[11]  Todd P. Coleman,et al.  Equivalence between minimal generative model graphs and directed information graphs , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[12]  D. Materassi Reconstruction of topologies for acyclic networks of dynamical systems , 2011, Proceedings of the 2011 American Control Conference.

[13]  Todd P. Coleman,et al.  A minimal approach to causal inference on topologies with bounded indegree , 2011, IEEE Conference on Decision and Control and European Control Conference.

[14]  Murti V. Salapaka,et al.  On the Problem of Reconstructing an Unknown Topology via Locality Properties of the Wiener Filter , 2010, IEEE Transactions on Automatic Control.

[15]  Murti V. Salapaka,et al.  Network reconstruction of dynamical polytrees with unobserved nodes , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[16]  Todd P. Coleman,et al.  Learning minimal latent directed information trees , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[17]  Ali Pinar,et al.  Optimal bounded-degree approximations of joint distributions of networks of stochastic processes , 2013, 2013 IEEE International Symposium on Information Theory.

[18]  Todd P. Coleman,et al.  Robust directed tree approximations for networks of stochastic processes , 2013, 2013 IEEE International Symposium on Information Theory.

[19]  Todd P. Coleman,et al.  Directed Information Graphs , 2012, IEEE Transactions on Information Theory.