Temporal Pattern Detection in Time-Varying Graphical Models

Graphical models allow to describe the interplay among variables of a system through a compact representation, suitable when relations evolve over time. For example, in a biological setting, genes interact differently depending on external environmental or metabolic factors. To incorporate this dynamics a viable strategy is to estimate a sequence of temporally related graphs assuming similarity among samples in different time points. While adjacent time points may direct the analysis towards a robust estimate of the underlying graph, the resulting model will not incorporate long-term or recurrent temporal relationships. In this work we propose a dynamical network inference model that leverages on kernels to consider general temporal patterns (such as circadian rhythms or seasonality). We show how our approach may also be exploited when the recurrent patterns are unknown, by coupling the network inference with a clustering procedure that detects possibly non-consecutive similar networks. Such clusters are then used to build similarity kernels. The convexity of the functional is determined by whether we impose or infer the kernel. In the first case, the optimisation algorithm exploits efficiently proximity operators with closed-form solutions. In the other case, we resort to an alternating minimisation procedure which jointly learns the temporal kernel and the underlying network. Extensive analysis on synthetic data shows the efficacy of our models compared to state-of-the-art methods. Finally, we applied our approach on two realworld applications to show how considering long-term patterns is fundamental to have insights on the behaviour of a complex system.

[1]  Shiqian Ma,et al.  Alternating Direction Methods for Latent Variable Gaussian Graphical Model Selection , 2012, Neural Computation.

[2]  E Bianco-Martinez,et al.  Successful network inference from time-series data using mutual information rate. , 2016, Chaos.

[3]  Andrew Gordon Wilson,et al.  Generalised Wishart Processes , 2010, UAI.

[4]  Amos J. Storkey,et al.  Bayesian Inference in Sparse Gaussian Graphical Models , 2013, ArXiv.

[5]  Sargur N. Srihari,et al.  Probabilistic graphical models in modern social network analysis , 2015, Social Network Analysis and Mining.

[6]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[7]  Genevera I. Allen,et al.  Graphical Models and Dynamic Latent Factors for Modeling Functional Brain Connectivity , 2019, 2019 IEEE Data Science Workshop (DSW).

[8]  Michael I. Jordan Graphical Models , 2003 .

[9]  Aram Galstyan,et al.  Efficient Covariance Estimation from Temporal Data , 2019, ArXiv.

[10]  Stephen P. Boyd,et al.  Network Inference via the Time-Varying Graphical Lasso , 2017, KDD.

[11]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[12]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[13]  Robert D. Nowak,et al.  Causal Network Inference Via Group Sparse Regularization , 2011, IEEE Transactions on Signal Processing.

[14]  Patrick Danaher,et al.  The joint graphical lasso for inverse covariance estimation across multiple classes , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[15]  Vincent Y. F. Tan,et al.  Learning Latent Tree Graphical Models , 2010, J. Mach. Learn. Res..

[16]  Pablo A. Parrilo,et al.  Latent variable graphical model selection via convex optimization , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[17]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[18]  Le Song,et al.  Evolving Cluster Mixed-Membership Blockmodel for Time-Evolving Networks , 2011, AISTATS.

[19]  E. Levina,et al.  Joint estimation of multiple graphical models. , 2011, Biometrika.

[20]  Federico Tomasi,et al.  Forward-Backward Splitting for Time-Varying Graphical Models , 2018, PGM.

[21]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[22]  Stephen P. Boyd,et al.  Toeplitz Inverse Covariance-Based Clustering of Multivariate Time Series Data , 2017, KDD.

[23]  Annette M. Molinaro,et al.  Prediction error estimation: a comparison of resampling methods , 2005, Bioinform..

[24]  Julia Hirschberg,et al.  V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure , 2007, EMNLP.