Latent Variable Time-varying Network Inference

In many applications of finance, biology and sociology, complex systems involve entities interacting with each other. These processes have the peculiarity of evolving over time and of comprising latent factors, which influence the system without being explicitly measured. In this work we present latent variable time-varying graphical lasso (LTGL), a method for multivariate time-series graphical modelling that considers the influence of hidden or unmeasurable factors. The estimation of the contribution of the latent factors is embedded in the model which produces both sparse and low-rank components for each time point. In particular, the first component represents the connectivity structure of observable variables of the system, while the second represents the influence of hidden factors, assumed to be few with respect to the observed variables. Our model includes temporal consistency on both components, providing an accurate evolutionary pattern of the system. We derive a tractable optimisation algorithm based on alternating direction method of multipliers, and develop a scalable and efficient implementation which exploits proximity operators in closed form. LTGL is extensively validated on synthetic data, achieving optimal performance in terms of accuracy, structure learning and scalability with respect to ground truth and state-of-the-art methods for graphical inference. We conclude with the application of LTGL to real case studies, from biology and finance, to illustrate how our method can be successfully employed to gain insights on multivariate time-series data.

[1]  Sargur N. Srihari,et al.  Probabilistic graphical models in modern social network analysis , 2015, Social Network Analysis and Mining.

[2]  Tim Beißbarth,et al.  Boolean ErbB network reconstructions and perturbation simulations reveal individual drug response in different breast cancer cell lines , 2014, BMC Systems Biology.

[3]  R. Tibshirani,et al.  Sparse estimation of a covariance matrix. , 2011, Biometrika.

[4]  Amos J. Storkey,et al.  Bayesian Inference in Sparse Gaussian Graphical Models , 2013, ArXiv.

[5]  C. Ding,et al.  On the Equivalence of Nonnegative Matrix Factorization and K-means - Spectral Clustering , 2005 .

[6]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[7]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[8]  Stephen P. Boyd,et al.  Network Lasso: Clustering and Optimization in Large Graphs , 2015, KDD.

[9]  Serena Ng,et al.  Evaluating latent and observed factors in macroeconomics and finance , 2006 .

[10]  Chao Sima,et al.  Inference of Gene Regulatory Networks Using Time-Series Data: A Survey , 2009, Current genomics.

[11]  M. Wainwright,et al.  HIGH-DIMENSIONAL COVARIANCE ESTIMATION BY MINIMIZING l1-PENALIZED LOG-DETERMINANT DIVERGENCE BY PRADEEP RAVIKUMAR , 2009 .

[12]  Lei Huang,et al.  Inference of protein-protein interaction networks from multiple heterogeneous data , 2015, EURASIP J. Bioinform. Syst. Biol..

[13]  Alfred O. Hero,et al.  Learning Latent Variable Gaussian Graphical Models , 2014, ICML.

[14]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[15]  Michael Hecker,et al.  Gene regulatory network inference: Data integration in dynamic models - A review , 2009, Biosyst..

[16]  Weiqing Wang,et al.  Perturbation Biology: Inferring Signaling Networks in Cellular Systems , 2013, PLoS Comput. Biol..

[17]  Shiqian Ma,et al.  Alternating Direction Methods for Latent Variable Gaussian Graphical Model Selection , 2012, Neural Computation.

[18]  Naoki Abe,et al.  Grouped graphical Granger modeling for gene expression regulatory networks discovery , 2009, Bioinform..

[19]  Adel Javanmard,et al.  Learning Linear Bayesian Networks with Latent Variables , 2012, ICML.

[20]  Annette M. Molinaro,et al.  Prediction error estimation: a comparison of resampling methods , 2005, Bioinform..

[21]  Venkat Chandrasekaran,et al.  Gaussian Multiresolution Models: Exploiting Sparse Markov and Covariance Structure , 2010, IEEE Transactions on Signal Processing.

[22]  Vincent Y. F. Tan,et al.  Learning Latent Tree Graphical Models , 2010, J. Mach. Learn. Res..

[23]  Amr Ahmed,et al.  Recovering time-varying networks of dependencies in social and biological studies , 2009, Proceedings of the National Academy of Sciences.

[24]  J. Selbig,et al.  Metabolomic and transcriptomic stress response of Escherichia coli , 2010, Molecular systems biology.

[25]  Robert D. Nowak,et al.  Causal Network Inference Via Group Sparse Regularization , 2011, IEEE Transactions on Signal Processing.

[26]  Seungyeop Han,et al.  Structured Learning of Gaussian Graphical Models , 2012, NIPS.

[27]  Alex J. Gibberd,et al.  Multiple Changepoint Estimation in High-Dimensional Gaussian Graphical Models , 2017, 1712.05786.

[28]  Kilian Q. Weinberger,et al.  Graph Laplacian Regularization for Large-Scale Semidefinite Programming , 2006, NIPS.

[29]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[30]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[31]  Ali Jalali,et al.  Learning the Dependence Graph of Time Series with Latent Factors , 2011, ICML.

[32]  Fang Han,et al.  Transelliptical Graphical Models , 2012, NIPS.

[33]  E Bianco-Martinez,et al.  Successful network inference from time-series data using mutual information rate. , 2016, Chaos.

[34]  Stephen P. Boyd,et al.  Network Inference via the Time-Varying Graphical Lasso , 2017, KDD.

[35]  Martin J. Wainwright.  Discussion: Latent variable graphical model selection via convex optimization , 2012, ArXiv.

[36]  R. Albert.  Network Inference, Analysis, and Modeling in Systems Biology , 2007, The Plant Cell Online.

[37]  R. Tibshirani,et al.  Covariance‐regularized regression and classification for high dimensional problems , 2009, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[38]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[39]  A. Willsky,et al.  Latent variable graphical model selection via convex optimization , 2010 .

[40]  Patrick Danaher,et al.  The joint graphical lasso for inverse covariance estimation across multiple classes , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.