Topology Identification of Directed Graphs via Joint Diagonalization of Correlation Matrices

Discovering connectivity patterns of directed networks is a crucial step to understand complex systems such as brain-, social-, and financial networks. Several existing network topology inference approaches rely on structural equation models (SEMs). These presume that exogenous inputs are available, which may be unrealistic in certain applications. Recently, an alternative line of work reformulated SEM-based topology identification as a three-way tensor decomposition task. This way, knowing the exogenous input correlation statistics (rather than the exogenous inputs themselves) suffices for network topology identification. The downside is that this approach is computationally expensive. In addition, it is hard to incorporate prior information of the network structure (e.g., sparsity and local smoothness) into this framework, while such prior information may help enhance performance when handling real-world noisy data. The present work puts forth a joint diagonalizaition (JD)-based approach to directed network topology inference. JD can be viewed as a variant of tensor decomposition, but features more efficient algorithms, and can readily account for the network structure. Different from existing alternatives, novel identifiability guarantees are derived regardless of the exogenous inputs or their statistics. Three JD algorithms tailored for network topology inference are developed, and their performance is showcased using simulated and real data tests.

[1]  Georgios B. Giannakis,et al.  Distributed Spectrum Sensing for Cognitive Radio Networks by Exploiting Sparsity , 2010, IEEE Transactions on Signal Processing.

[2]  Pascal Frossard,et al.  Learning Graphs From Data: A Signal Representation Perspective , 2018, IEEE Signal Processing Magazine.

[3]  J. Kruskal Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[4]  Georgios B. Giannakis,et al.  Tensor Decompositions for Identifying Directed Graph Topologies and Tracking Dynamic Networks , 2016, IEEE Transactions on Signal Processing.

[5]  Nikos D. Sidiropoulos,et al.  Tensor Decomposition for Signal Processing and Machine Learning , 2016, IEEE Transactions on Signal Processing.

[6]  Wotao Yin,et al.  A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion , 2013, SIAM J. Imaging Sci..

[7]  D. Le Bihan,et al.  Diffusion tensor imaging: Concepts and applications , 2001, Journal of magnetic resonance imaging : JMRI.

[8]  Nikos D. Sidiropoulos,et al.  Directed network topology inference via sparse joint diagonalization , 2017, 2017 51st Asilomar Conference on Signals, Systems, and Computers.

[9]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[10]  A. Goldberger STRUCTURAL EQUATION METHODS IN THE SOCIAL SCIENCES , 1972 .

[11]  Eric D. Kolaczyk,et al.  Statistical Analysis of Network Data: Methods and Models , 2009 .

[12]  Bernhard Schölkopf,et al.  Uncovering the Temporal Dynamics of Diffusion Networks , 2011, ICML.

[13]  Gonzalo Mateos,et al.  Proximal-Gradient Algorithms for Tracking Cascades Over Social Networks , 2014, IEEE Journal of Selected Topics in Signal Processing.

[14]  A. Stegeman,et al.  On Kruskal's uniqueness condition for the Candecomp/Parafac decomposition , 2007 .

[15]  D. Kaplan Structural Equation Modeling: Foundations and Extensions , 2000 .

[16]  Zhi-Quan Luo,et al.  A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization , 2012, SIAM J. Optim..

[17]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[18]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[19]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[20]  Alejandro Ribeiro,et al.  Consensus in Ad Hoc WSNs With Noisy Links—Part I: Distributed Estimation of Deterministic Signals , 2008, IEEE Transactions on Signal Processing.

[21]  Christos Faloutsos,et al.  Kronecker Graphs: An Approach to Modeling Networks , 2008, J. Mach. Learn. Res..

[22]  Klaus Obermayer,et al.  Quadratic optimization for simultaneous matrix diagonalization , 2006, IEEE Transactions on Signal Processing.

[23]  Santiago Segarra,et al.  Network Topology Inference from Spectral Templates , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[24]  Georgios B. Giannakis,et al.  Inferring directed network topologies via tensor factorization , 2016, 2016 50th Asilomar Conference on Signals, Systems and Computers.

[25]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[26]  Georgios B. Giannakis,et al.  Identifiability of sparse structural equation models for directed and cyclic networks , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[27]  M. Wax,et al.  A least-squares approach to joint diagonalization , 1997, IEEE Signal Processing Letters.

[28]  Santiago Segarra,et al.  Connecting the Dots: Identifying Network Structure via Graph Signal Processing , 2018, IEEE Signal Processing Magazine.

[29]  Georgios B. Giannakis,et al.  Inference of Gene Regulatory Networks with Sparse Structural Equation Models Exploiting Genetic Perturbations , 2013, PLoS Comput. Biol..

[30]  Eric Moreau,et al.  A generalization of joint-diagonalization criteria for source separation , 2001, IEEE Trans. Signal Process..

[31]  Bijan Afsari,et al.  Sensitivity Analysis for the Problem of Matrix Joint Diagonalization , 2008, SIAM J. Matrix Anal. Appl..

[32]  Arie Yeredor,et al.  Non-orthogonal joint diagonalization in the least-squares sense with application in blind source separation , 2002, IEEE Trans. Signal Process..

[33]  Georgios B. Giannakis,et al.  Topology Identification and Learning over Graphs: Accounting for Nonlinearities and Dynamics , 2018, Proceedings of the IEEE.

[34]  P. Tichavsky,et al.  Fast Approximate Joint Diagonalization Incorporating Weight Matrices , 2009, IEEE Transactions on Signal Processing.

[35]  Gonzalo Mateos,et al.  Dynamic structural equation models for tracking topologies of social networksy , 2013, 2013 5th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[36]  Andreas Ziehe,et al.  A Fast Algorithm for Joint Diagonalization with Non-orthogonal Transformations and its Application to Blind Source Separation , 2004, J. Mach. Learn. Res..

[37]  Morteza Mardani,et al.  Subspace Learning and Imputation for Streaming Big Data Matrices and Tensors , 2014, IEEE Transactions on Signal Processing.