Estimating networks with jumps.

We study the problem of estimating a temporally varying coefficient and varying structure (VCVS) graphical model underlying data collected over a period of time, such as social states of interacting individuals or microarray expression profiles of gene networks, as opposed to i.i.d. data from an invariant model widely considered in current literature of structural estimation. In particular, we consider the scenario in which the model evolves in a piece-wise constant fashion. We propose a procedure that estimates the structure of a graphical model by minimizing the temporally smoothed L1 penalized regression, which allows jointly estimating the partition boundaries of the VCVS model and the coefficient of the sparse precision matrix on each block of the partition. A highly scalable proximal gradient method is proposed to solve the resultant convex optimization problem; and the conditions for sparsistent estimation and the convergence rate of both the partition boundaries and the network structure are established for the first time for such estimators.

[1]  P. Perron,et al.  Estimating and testing linear models with multiple structural changes , 1995 .

[2]  E. Levina,et al.  Joint Structure Estimation for Categorical Markov Networks , 2010 .

[3]  Z. Harchaoui,et al.  Multiple Change-Point Estimation With a Total Variation Penalty , 2010 .

[4]  Pei Wang,et al.  Partial Correlation Estimation by Joint Sparse Regression Models , 2008, Journal of the American Statistical Association.

[5]  Larry A. Wasserman,et al.  Time varying undirected graphs , 2008, Machine Learning.

[6]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[7]  S. Geer,et al.  Locally adaptive regression splines , 1997 .

[8]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[9]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[10]  Adam J. Rothman,et al.  Sparse permutation invariant covariance estimation , 2008, 0801.4837.

[11]  E. Levina,et al.  Joint estimation of multiple graphical models. , 2011, Biometrika.

[12]  S. Szarek,et al.  Chapter 8 - Local Operator Theory, Random Matrices and Banach Spaces , 2001 .

[13]  Hongzhe Li,et al.  Gradient directed regularization for sparse Gaussian concentration graphs, with applications to inference of genetic networks. , 2006, Biostatistics.

[14]  I JordanMichael,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008 .

[15]  Eric P. Xing,et al.  On Sparse Nonparametric Conditional Covariance Selection , 2010, ICML.

[16]  A. Rinaldo.  Properties and refinements of the fused lasso , 2008, 0805.0234.

[17]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[18]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[19]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[20]  Eric P. Xing,et al.  Sparsistent Estimation of Time-Varying Discrete Markov Random Fields , 2009, 0907.2337.

[21]  Massimiliano Pontil,et al.  Taking Advantage of Sparsity in Multi-Task Learning , 2009, COLT.

[22]  S. Geer,et al.  On the conditions used to prove oracle results for the Lasso , 2009, 0910.0722.

[23]  P. Brucker.  Review of recent development: An O( n) algorithm for quadratic knapsack problems , 1984 .

[24]  Stanley R. Johnson,et al.  Varying Coefficient Models , 1984 .

[25]  Le Song,et al.  Estimating time-varying networks , 2008, ISMB 2008.

[26]  J. Lafferty,et al.  High-dimensional Ising model selection using ℓ1-regularized logistic regression , 2010, 1010.0311.

[27]  J. Zidek,et al.  ON SEGMENTED MULTIVARIATE REGRESSION , 1997 .

[28]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..

[29]  F. Bunea.  Honest variable selection in linear and logistic regression models via $\ell_1$ and $\ell_1+\ell_2$ penalization , 2008, 0808.4051.

[30]  Bin Yu,et al.  High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence , 2008, 0811.3628.

[31]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[32]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[33]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[34]  Y. Nesterov.  Gradient methods for minimizing composite objective function , 2007 .

[35]  Jianqing Fan,et al.  NETWORK EXPLORATION VIA THE ADAPTIVE LASSO AND SCAD PENALTIES. , 2009, The annals of applied statistics.

[36]  Amr Ahmed,et al.  Recovering time-varying networks of dependencies in social and biological studies , 2009, Proceedings of the National Academy of Sciences.

[37]  Martin J. Wainwright,et al.  Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$ -Constrained Quadratic Programming (Lasso) , 2009, IEEE Transactions on Information Theory.

[38]  Hansheng Wang,et al.  Nonparametric Covariance Model , 2008, Statistica Sinica.

[39]  Pei Wang,et al.  Learning networks from high dimensional binary data: An application to genomic instability data , 2009, 0908.3882.

[40]  Ben Taskar,et al.  Introduction to Statistical Relational Learning (Adaptive Computation and Machine Learning) , 2007 .

[41]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[42]  P. Bickel,et al.  Regularized estimation of large covariance matrices , 2008, 0803.1909.