Graph-Based Learning Under Perturbations via Total Least-Squares

Graphs are pervasive in different fields unveiling complex relationships between data. Two major graph-based learning tasks are topology identification and inference of signals over graphs. Among the possible models to explain data interdependencies, structural equation models (SEMs) accommodate a gamut of applications involving topology identification. Obtaining conventional SEMs though requires measurements across nodes. On the other hand, typical signal inference approaches ‘blindly trust’ a given nominal topology. In practice however, signal or topology perturbations may be present in both tasks, due to model mismatch, outliers, outages or adversarial behavior. To cope with such perturbations, this work introduces a regularized total least-squares (TLS) approach and iterative algorithms with convergence guarantees to solve both tasks. Further generalizations are also considered relying on structured and/or weighted TLS when extra prior information on the perturbation is available. Analyses with simulated and real data corroborate the effectiveness of the novel TLS-based approaches.

[1]  Alexander Jung,et al.  Learning conditional independence structure for high-dimensional uncorrelated vector processes , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Motoaki Kawanabe,et al.  Modeling Sparse Connectivity Between Underlying Brain Sources for EEG/MEG , 2009, IEEE Transactions on Biomedical Engineering.

[3]  Marc Teboulle,et al.  Finding a Global Optimal Solution for a Quadratically Constrained Fractional Quadratic Problem with Applications to the Regularized Total Least Squares , 2006, SIAM J. Matrix Anal. Appl..

[4]  J. Kruskal Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[5]  Christodoulos A. Floudas,et al.  Computational Experience with a New Class of Convex Underestimators: Box-constrained NLP Problems , 2004, J. Glob. Optim..

[6]  Zhaohui S. Qin,et al.  A second generation human haplotype map of over 3.1 million SNPs , 2007, Nature.

[7]  Georgios B. Giannakis,et al.  Sparsity-Cognizant Total Least-Squares for Perturbed Compressive Sampling , 2010, IEEE Transactions on Signal Processing.

[8]  Sergio Barbarossa,et al.  Robust Graph Signal Processing in the Presence of Uncertainties on Graph Topology , 2018, 2018 IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[9]  Sabine Van Huffel,et al.  Overview of total least-squares methods , 2007, Signal Process..

[10]  Christos Faloutsos,et al.  Kronecker Graphs: An Approach to Modeling Networks , 2008, J. Mach. Learn. Res..

[11]  P. Tseng Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization , 2001 .

[12]  P. Sprent Models in regression and related topics , 1971 .

[13]  Eric D. Kolaczyk,et al.  On the Propagation of Low-Rate Measurement Error to Subgraph Counts in Large Networks , 2014, J. Mach. Learn. Res..

[14]  Sergio Barbarossa,et al.  Small Perturbation Analysis of Network Topologies , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[15]  Joseph K. Pickrell,et al.  Understanding mechanisms underlying human gene expression variation with RNA sequencing , 2010, Nature.

[16]  J. H. Wilkinson The algebraic eigenvalue problem , 1966 .

[17]  A. Goldberger STRUCTURAL EQUATION METHODS IN THE SOCIAL SCIENCES , 1972 .

[18]  M. Aoki,et al.  On a priori error estimates of some identification methods , 1970 .

[19]  Larry A. Wasserman,et al.  Time varying undirected graphs , 2008, Machine Learning.

[20]  Alexander Jung,et al.  Learning the Conditional Independence Structure of Stationary Time Series: A Multitask Learning Approach , 2014, IEEE Transactions on Signal Processing.

[21]  Paolo Di Lorenzo,et al.  Online Recovery of Time- varying Signals Defined over Dynamic Graphs , 2018, 2018 26th European Signal Processing Conference (EUSIPCO).

[22]  Sunil K. Narang,et al.  Localized iterative methods for interpolation in graph structured data , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[23]  M. Berry,et al.  Solving total least-squares problems in information retrieval , 2000 .

[24]  Eric D. Kolaczyk,et al.  Estimation of edge density in noisy networks , 2018 .

[25]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[26]  Georgios B. Giannakis,et al.  Topology Identification and Learning over Graphs: Accounting for Nonlinearities and Dynamics , 2018, Proceedings of the IEEE.

[27]  Santiago Segarra,et al.  Inference of Graph Topology , 2018 .

[28]  Georgios B. Giannakis,et al.  Inference of Spatio-Temporal Functions Over Graphs via Multikernel Kriged Kalman Filtering , 2017, IEEE Transactions on Signal Processing.

[29]  Sabine Van Huffel,et al.  Total least squares problem - computational aspects and analysis , 1991, Frontiers in applied mathematics.

[30]  Gang Wang,et al.  Ergodic Energy Management Leveraging Resource Variability in Distribution Grids , 2015, IEEE Transactions on Power Systems.

[31]  Georgios B. Giannakis,et al.  Network topology inference via elastic net structural equation models , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[32]  Georgios B. Giannakis,et al.  Distributed consensus-based demodulation: algorithms and error analysis , 2010, IEEE Transactions on Wireless Communications.

[33]  Robert D. Nowak,et al.  Causal Network Inference Via Group Sparse Regularization , 2011, IEEE Transactions on Signal Processing.

[34]  B. Moor Structured total least squares and L2 approximation problems , 1993 .

[35]  Georgios B. Giannakis,et al.  Semi-Blind Inference of Topologies and Dynamical Processes Over Dynamic Graphs , 2018, IEEE Transactions on Signal Processing.

[36]  Mark Crovella,et al.  Network Kriging , 2005, IEEE Journal on Selected Areas in Communications.

[37]  George Kollios,et al.  Clustering Large Probabilistic Graphs , 2013, IEEE Transactions on Knowledge and Data Engineering.

[38]  Alexander Jung,et al.  Graphical LASSO based Model Selection for Time Series , 2014, IEEE Signal Processing Letters.

[39]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[40]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[41]  Georgios B. Giannakis,et al.  Inference of Gene Regulatory Networks with Sparse Structural Equation Models Exploiting Genetic Perturbations , 2013, PLoS Comput. Biol..

[42]  Georgios B. Giannakis,et al.  Identifiability of sparse structural equation models for directed and cyclic networks , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[43]  Geert Leus,et al.  Filtering Random Graph Processes Over Random Time-Varying Graphs , 2017, IEEE Transactions on Signal Processing.

[44]  S. Lauritzen The EM algorithm for graphical association models with missing data , 1995 .

[45]  Antonio Ortega,et al.  Submitted to Ieee Transactions on Signal Processing 1 Efficient Sampling Set Selection for Bandlimited Graph Signals Using Graph Spectral Proxies , 2022 .

[46]  Gonzalo Mateos,et al.  Proximal-Gradient Algorithms for Tracking Cascades Over Social Networks , 2014, IEEE Journal of Selected Topics in Signal Processing.

[47]  D. Kaplan Structural Equation Modeling: Foundations and Extensions , 2000 .

[48]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[49]  Georgios B. Giannakis,et al.  Kernel-based Inference of Functions over Graphs , 2017, ArXiv.

[50]  George Karypis,et al.  SLIM: Sparse Linear Methods for Top-N Recommender Systems , 2011, 2011 IEEE 11th International Conference on Data Mining.

[51]  Georgios B. Giannakis,et al.  Tensor Decompositions for Identifying Directed Graph Topologies and Tracking Dynamic Networks , 2016, IEEE Transactions on Signal Processing.

[52]  Sergio Barbarossa,et al.  Signal and Graph Perturbations via Total Least-Squares , 2018, 2018 52nd Asilomar Conference on Signals, Systems, and Computers.

[53]  I. Stancu-Minasian Nonlinear Fractional Programming , 1997 .

[54]  B. Muthén A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators , 1984 .

[55]  Sergio Barbarossa,et al.  Sampling and Recovery of Graph Signals , 2017, 1712.09310.

[56]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .