Bayesian Topology Learning and noise removal from network data

Learning the topology of a graph from available data is of great interest in many emerging applications. Some examples are social networks, internet of things networks (intelligent IoT and industrial IoT), biological connection networks, sensor networks and traffic network patterns. In this paper, a graph topology inference approach is proposed to learn the underlying graph structure from a given set of noisy multi-variate observations, which are modeled as graph signals generated from a Gaussian Markov Random Field (GMRF) process. A factor analysis model is applied to represent the graph signals in a latent space where the basis is related to the underlying graph structure. An optimal graph filter is also developed to recover the graph signals from noisy observations. In the final step, an optimization problem is proposed to learn the underlying graph topology from the recovered signals. Moreover, a fast algorithm employing the proximal point method has been proposed to solve the problem efficiently. Experimental results employing both synthetic and real data show the effectiveness of the proposed method in recovering the signals and inferring the underlying graph.

[1]  Steven Kay,et al.  Fundamentals Of Statistical Signal Processing , 2001 .

[2]  Lida Xu,et al.  The internet of things: a survey , 2014, Information Systems Frontiers.

[3]  Mark Sanderson,et al.  Christopher D. Manning, Prabhakar Raghavan, Hinrich Schütze, Introduction to Information Retrieval, Cambridge University Press 2008. ISBN-13 978-0-521-86571-5, xxi + 482 pages , 2010, Natural Language Engineering.

[4]  Antonio Ortega,et al.  Graph Learning From Data Under Laplacian and Structural Constraints , 2016, IEEE Journal of Selected Topics in Signal Processing.

[5]  Luiz A. Baccalá,et al.  Partial directed coherence: a new concept in neural structure determination , 2001, Biological Cybernetics.

[6]  Jelena Kovacevic,et al.  Graph topology recovery for regular and irregular graphs , 2017, 2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[7]  Kim-Chuan Toh,et al.  Solving Log-Determinant Optimization Problems by a Newton-CG Primal Proximal Point Algorithm , 2010, SIAM J. Optim..

[8]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[9]  Pascal Frossard,et al.  Learning Graphs From Data: A Signal Representation Perspective , 2018, IEEE Signal Processing Magazine.

[10]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[11]  J. E. Jackson,et al.  Statistical Factor Analysis and Related Methods: Theory and Applications , 1995 .

[12]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[13]  Robert D. Nowak,et al.  Causal Network Inference Via Group Sparse Regularization , 2011, IEEE Transactions on Signal Processing.

[14]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[15]  Gregory Dudek,et al.  A practical algorithm for network topology inference , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[16]  Lieven Vandenberghe,et al.  Topology Selection in Graphical Models of Autoregressive Processes , 2010, J. Mach. Learn. Res..

[17]  H. Vincent Poor,et al.  Graph Topology Learning and Signal Recovery Via Bayesian Inference , 2019, 2019 IEEE Data Science Workshop (DSW).

[18]  Éric Gaussier,et al.  A Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation , 2005, ECIR.

[19]  Nikolas P. Galatsanos,et al.  Methods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation , 1992, IEEE Trans. Image Process..

[20]  Johan Efberg,et al.  YALMIP : A toolbox for modeling and optimization in MATLAB , 2004 .

[21]  Mona Azadkia Adaptive Estimation of Noise Variance and Matrix Estimation via USVT Algorithm , 2018, 1801.10015.

[22]  Karl Skretting,et al.  Topology Inference and Signal Representation Using Dictionary Learning , 2019, 2019 27th European Signal Processing Conference (EUSIPCO).

[23]  José M. F. Moura,et al.  Signal denoising on graphs via graph filtering , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[24]  Eduardo Pavez,et al.  Learning Graphs With Monotone Topology Properties and Multiple Connected Components , 2017, IEEE Transactions on Signal Processing.

[25]  Mahmoud Ramezani-Mayiami JOINT TOPOLOGY LEARNING AND GRAPH SIGNAL RECOVERY VIA KALMAN FILTER IN CAUSAL DATA PROCESSES , 2018, 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP).

[26]  Santiago Segarra,et al.  Connecting the Dots: Identifying Network Structure via Graph Signal Processing , 2018, IEEE Signal Processing Magazine.

[27]  Mahmoud Ramezani-Mayiami Joint Graph Learning and Signal Recovery via Kalman Filter for Multivariate Auto-Regressive Processes , 2018, 2018 26th European Signal Processing Conference (EUSIPCO).

[28]  Pierre Vandergheynst,et al.  GSPBOX: A toolbox for signal processing on graphs , 2014, ArXiv.

[29]  Rainer Goebel,et al.  Investigating directed cortical interactions in time-resolved fMRI data using vector autoregressive modeling and Granger causality mapping. , 2003, Magnetic resonance imaging.

[30]  Pascal Frossard,et al.  Learning Laplacian Matrix in Smooth Graph Signal Representations , 2014, IEEE Transactions on Signal Processing.

[31]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[32]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[33]  Tharam S. Dillon,et al.  Noise Removal in the Presence of Significant Anomalies for Industrial IoT Sensor Data in Manufacturing , 2020, IEEE Internet of Things Journal.

[34]  D. Varberg,et al.  Another Proof that Convex Functions are Locally Lipschitz , 1974 .

[35]  Santiago Segarra,et al.  Network Topology Inference from Spectral Templates , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[36]  J. Neumann Zur Theorie der Gesellschaftsspiele , 1928 .

[37]  Vassilis Kalofolias,et al.  How to Learn a Graph from Smooth Signals , 2016, AISTATS.

[38]  Sergio Barbarossa,et al.  Graph Topology Inference Based on Sparsifying Transform Learning , 2018, IEEE Transactions on Signal Processing.

[39]  A. Basilevsky Statistical Factor Analysis and Related Methods: Theory and Applications , 1994 .

[40]  José M. F. Moura,et al.  Signal inpainting on graphs via total variation minimization , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[41]  Karl J. Friston Functional and effective connectivity in neuroimaging: A synthesis , 1994 .

[42]  Ieee Staff,et al.  2013 IEEE Global Conference on Signal and Information Processing (GlobalSIP) , 2013 .

[43]  Karl Skretting,et al.  Robust Graph Topology Learning and Application in Stock Market Inference , 2019, 2019 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[44]  G. Giannakis,et al.  Nonlinear Structural Vector Autoregressive Models for Inferring Effective Brain Network Connectivity , 2016, 1610.06551.

[45]  Georgios B. Giannakis,et al.  Kernel-Based Structural Equation Models for Topology Identification of Directed Networks , 2016, IEEE Transactions on Signal Processing.

[46]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[47]  Baltasar Beferull-Lozano,et al.  Graph recursive least squares filter for topology inference in causal data processes , 2017, 2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[48]  José M. F. Moura,et al.  Signal Processing on Graphs: Modeling (Causal) Relations in Big Data , 2015, ArXiv.

[49]  Joshua B. Tenenbaum,et al.  Discovering Structure by Learning Sparse Graphs , 2010 .

[50]  Georgios B. Giannakis,et al.  Network topology inference via elastic net structural equation models , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[51]  Hyunjoong Kim,et al.  Functional Analysis I , 2017 .

[52]  Peter D. Wentzell,et al.  Measurement errors in multivariate chemical data , 2013 .

[53]  Alfred O. Hero,et al.  Learning sparse graphs under smoothness prior , 2016, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[54]  Heinz H. Bauschke,et al.  Joint minimization with alternating Bregman proximity operators , 2005 .

[55]  Ramkrishna Pasumarthy,et al.  Identifying Topology of Power Distribution Networks Based on Smart Meter Data , 2016, ArXiv.

[56]  Santiago Segarra,et al.  Network topology inference from non-stationary graph signals , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[57]  Pascal Frossard,et al.  Learning Heat Diffusion Graphs , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[58]  Daniel Pérez Palomar,et al.  Structured Graph Learning Via Laplacian Spectral Constraints , 2019, NeurIPS.

[59]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[60]  Georgios B. Giannakis,et al.  Topology Identification and Learning over Graphs: Accounting for Nonlinearities and Dynamics , 2018, Proceedings of the IEEE.

[61]  Michael G. Rabbat,et al.  Characterization and Inference of Graph Diffusion Processes From Observations of Stationary Signals , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[62]  Georgios B. Giannakis,et al.  Tracking Switched Dynamic Network Topologies From Information Cascades , 2016, IEEE Transactions on Signal Processing.

[63]  J. Moreau Proximité et dualité dans un espace hilbertien , 1965 .

[64]  Технология,et al.  National Climatic Data Center , 2011 .

[65]  Michael Elad,et al.  Dual Graph Regularized Dictionary Learning , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[66]  Quanzheng Li,et al.  A Spectral Graph Regression Model for Learning Brain Connectivity of Alzheimer’s Disease , 2015, PloS one.

[67]  Eduardo Pavez,et al.  Graph Learning From Filtered Signals: Graph System and Diffusion Kernel Identification , 2018, IEEE Transactions on Signal and Information Processing over Networks.

[68]  Antonio Ortega,et al.  Generalized Laplacian precision matrix estimation for graph signal processing , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[69]  Katya Scheinberg,et al.  Learning Sparse Gaussian Markov Networks Using a Greedy Coordinate Ascent Approach , 2010, ECML/PKDD.

[70]  Pierre Vandergheynst,et al.  Graph Signal Processing: Overview, Challenges, and Applications , 2017, Proceedings of the IEEE.

[71]  Michael R. Lyu,et al.  Mining social networks using heat diffusion processes for marketing candidates selection , 2008, CIKM '08.

[72]  Pascal Frossard,et al.  Learning time varying graphs , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[73]  José M. F. Moura,et al.  Signal Processing on Graphs: Causal Modeling of Unstructured Data , 2015, IEEE Transactions on Signal Processing.

[74]  Johan Löfberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004 .

[75]  Zhi Ding,et al.  Introducing Hypergraph Signal Processing: Theoretical Foundation and Practical Applications , 2019, IEEE Internet of Things Journal.