Blind Learning of Tree Network Topologies in the Presence of Hidden Nodes

This paper considers the problem of learning the unknown structure of a network with the underlying topology given by a polyforest (a collection of directed trees with potentially multiple roots). The main result is an algorithm that consistently learns the network structure using only second-order statistics of the data. The methodology is robust with respect to the presence of unmeasured (latent) nodes: the algorithm detects the exact number and location of the latent nodes, when they satisfy specific degree conditions in the actual network graph. It is shown that the same degree conditions are also necessary for a consistent reconstruction. Thus, the proposed reconstruction algorithm achieves the fundamental limitations in learning the structure of a polyforest network of linear dynamic systems in the presence of latent nodes. This paper overcomes the limitations of previous results that only addressed single-rooted trees, tackling the problem in an efficient way since the computational complexity of the derived algorithm is proven to be polynomial in the number of observed nodes.

[1]  Stan Matwin,et al.  A formal approach to using data distributions for building causal polytree structures , 2004, Inf. Sci..

[2]  P. Spirtes,et al.  Ancestral graph Markov models , 2002 .

[3]  Murti V. Salapaka,et al.  Reconstruction of directed acyclic networks of dynamical systems , 2013, 2013 American Control Conference.

[4]  Tengfei Liu,et al.  A Survey on Latent Tree Models and Applications , 2013, J. Artif. Intell. Res..

[5]  Ye Yuan,et al.  Network Identifiability from Intrinsic Noise , 2017, IEEE Transactions on Automatic Control.

[6]  Jorge Cortés,et al.  Network Identification With Latent Nodes via Autoregressive Models , 2016, IEEE Transactions on Control of Network Systems.

[7]  Tandy J. Warnow,et al.  A Few Logs Suffice to Build (almost) All Trees: Part II , 1999, Theor. Comput. Sci..

[8]  Renbin Xiao,et al.  Outer synchronization and parameter identification approach to the resilient recovery of supply network with uncertainty , 2017 .

[9]  Pierre Giot,et al.  Market Models: A Guide to Financial Data Analysis , 2003 .

[10]  Nam P. Nguyen,et al.  Sources of misinformation in Online Social Networks: Who to suspect? , 2012, MILCOM 2012 - 2012 IEEE Military Communications Conference.

[11]  Li Guo,et al.  Discovering Multiple Diffusion Source Nodes in Social Networks , 2014, ICCS.

[12]  Vincent Y. F. Tan,et al.  Learning Latent Tree Graphical Models , 2010, J. Mach. Learn. Res..

[13]  Murti V. Salapaka,et al.  Notions of Separation in Graphs of Dynamical Systems , 2014 .

[14]  Wen-Xu Wang,et al.  Detecting hidden nodes in complex networks from time series. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  Sean C. Warnick,et al.  Robust dynamical network structure reconstruction , 2011, Autom..

[16]  Donatello Materassi,et al.  Topological identification in networks of dynamical systems , 2008, 2008 47th IEEE Conference on Decision and Control.

[17]  Donatello Materassi,et al.  Inferring the structure of polytree networks of dynamic systems with hidden nodes , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[18]  Murti V. Salapaka,et al.  On the Problem of Reconstructing an Unknown Topology via Locality Properties of the Wiener Filter , 2010, IEEE Transactions on Automatic Control.

[19]  Saverio Bolognani,et al.  Identification of power distribution network topology via voltage correlation analysis , 2013, 52nd IEEE Conference on Decision and Control.

[20]  Sriram Vishwanath,et al.  One breaker is enough: Hidden topology attacks on power grids , 2015, 2015 IEEE Power & Energy Society General Meeting.

[21]  Tore Hägglund,et al.  Closed-Loop Anesthesia in Children using a PID Controller: A Pilot Study , 2012 .

[22]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[23]  Le Song,et al.  Spectral Methods for Learning Multivariate Latent Tree Structure , 2011, NIPS.

[24]  Sean C. Warnick,et al.  Passive reconstruction of non-target-specific discrete-time LTI systems , 2016, 2016 American Control Conference (ACC).

[25]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[26]  Xavier Bombois,et al.  Errors-in-variables identification in dynamic networks - Consistency results for an instrumental variable approach , 2015, Autom..

[27]  Todd P. Coleman,et al.  Learning Minimal Latent Directed Information Polytrees , 2016, Neural Computation.

[28]  Mark W. Schmidt,et al.  Learning Graphical Model Structure Using L1-Regularization Paths , 2007, AAAI.

[29]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[30]  Sean C. Warnick,et al.  Necessary and Sufficient Conditions for Dynamical Structure Reconstruction of LTI Networks , 2008, IEEE Transactions on Automatic Control.

[31]  D. Botstein,et al.  Cluster analysis and display of genome-wide expression patterns. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[32]  Sean C. Warnick,et al.  Dynamical structure function identifiability conditions enabling signal structure reconstruction , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[33]  Marc Timme,et al.  Revealing physical interaction networks from statistics of collective dynamics , 2017, Science Advances.

[34]  Steven Skiena,et al.  Lowest common ancestors in trees and directed acyclic graphs , 2005, J. Algorithms.

[35]  Uffe Kjaerulff.  HUGS: Combining Exact Inference and Gibbs Sampling in Junction Trees , 2011 .

[36]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[37]  Paul M.J. Van den Hof,et al.  On Representations of Linear Dynamic Networks , 2018 .

[38]  Lennart Ljung,et al.  Linear Dynamic Network Reconstruction from Heterogeneous Datasets , 2016, ArXiv.

[39]  Zhengtao Ding,et al.  A dynamic approach to identification of multiple harmonic sources in power distribution systems , 2016 .

[40]  E. B. Wilson.  Probable Inference, the Law of Succession, and Statistical Inference , 1927 .

[41]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[42]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[43]  Sean C. Warnick,et al.  Network semantics of dynamical systems , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[44]  Mario Sznaier.  Compressive Information Extraction: A Dynamical Systems Approach , 2012 .

[45]  Mattia Zorzi,et al.  AR Identification of Latent-Variable Graphical Models , 2014, IEEE Transactions on Automatic Control.

[46]  Wen-Xu Wang,et al.  Reconstructing propagation networks with natural diversity and identifying hidden sources , 2014, Nature Communications.

[47]  Daphne Koller,et al.  Using Learning for Approximation in Stochastic Processes , 1998, ICML.