Learning Dynamic Conditional Gaussian Graphical Models

In the paper, we propose a class of dynamic conditional Gaussian graphical models (DCGGMs) based on a set of non-identical distribution observations, which changes smoothly with time or condition. Specifically, the DCGGMs model the dynamic output network influenced by conditioning input variables, which are encoded by a set of varying parameters. Moreover, we propose a joint smooth graphical Lasso to estimate the DCGGMs, which combines kernel smoother with sparse group Lasso penalty. At the same time, we design an efficient accelerated proximal gradient algorithm to solve this estimator. Theoretically, we establish the asymptotic properties of our model on consistency and sparsistency under the high-dimensional settings. In particular, we highlight a class of consistency theory for dynamic graphical models, in which the sample size can be seen as <inline-formula><tex-math notation="LaTeX">$n^{4/5}$ </tex-math><alternatives><inline-graphic xlink:href="huang-ieq1-2777462.gif"/></alternatives></inline-formula> for estimating a local graphical model when the bandwidth parameter <inline-formula><tex-math notation="LaTeX">$h$ </tex-math><alternatives><inline-graphic xlink:href="huang-ieq2-2777462.gif"/></alternatives></inline-formula> of kernel smoother is chosen as <inline-formula><tex-math notation="LaTeX">$h\; \asymp\; n^{-1/5}$</tex-math> <alternatives><inline-graphic xlink:href="huang-ieq3-2777462.gif"/></alternatives></inline-formula> for describing the dynamic. Finally, the extensive numerical experiments on both synthetic and real datasets are provided to support the effectiveness of the proposed method.

[1]  Patrick Danaher,et al.  The joint graphical lasso for inverse covariance estimation across multiple classes , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[2]  C. O. A. D. P. R. M. A. E. Stimation.  Covariate Adjusted Precision Matrix Estimation with an Application in Genetical Genomics , 2011 .

[3]  R. Tibshirani,et al.  A note on the group lasso and a sparse group lasso , 2010, 1001.0736.

[4]  Mladen Kolar,et al.  Estimating networks with jumps. , 2010, Electronic journal of statistics.

[5]  Tristan Mary-Huard,et al.  Structured regularization for conditional Gaussian graphical models , 2014, Statistics and Computing.

[6]  Lan Xue,et al.  Variable Selection in High-dimensional Varying-coefficient Models with Global Optimality , 2012, J. Mach. Learn. Res..

[7]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[8]  Kyung-Ah Sohn,et al.  Joint Estimation of Structured Sparsity and Output Structure in Multiple-Output Regression via Inverse-Covariance Regularization , 2012, AISTATS.

[9]  Hongyu Zhao,et al.  Sparse Estimation of Conditional Graphical Models With Application to Gene Networks , 2012, Journal of the American Statistical Association.

[10]  Seyoung Kim,et al.  Correction: Learning Gene Networks under SNP Perturbations Using eQTL Datasets , 2014, PLoS Comput. Biol..

[11]  James D. B. Nelson,et al.  Regularized Estimation of Piecewise Constant Gaussian Graphical Models: The Group-Fused Graphical Lasso , 2015, 1512.06171.

[12]  Larry A. Wasserman,et al.  The huge Package for High-dimensional Undirected Graph Estimation in R , 2012, J. Mach. Learn. Res..

[13]  Feihu Huang,et al.  Joint Learning of Multiple Sparse Matrix Gaussian Graphical Models , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[14]  J. Zico Kolter,et al.  Sparse Gaussian Conditional Random Fields: Algorithms, Theory, and Application to Energy Forecasting , 2013, ICML.

[15]  T. Cai,et al.  A Constrained ℓ1 Minimization Approach to Sparse Precision Matrix Estimation , 2011, 1102.2233.

[16]  Harrison H. Zhou,et al.  Asymptotically Normal and Efficient Estimation of Covariate-Adjusted Gaussian Graphical Model , 2013, Journal of the American Statistical Association.

[17]  Yufeng Liu,et al.  Joint estimation of multiple precision matrices with common structures , 2015, J. Mach. Learn. Res..

[18]  Chenlei Leng,et al.  Dynamic Covariance Models , 2016 .

[19]  Pierre Pinson,et al.  Global Energy Forecasting Competition 2012 , 2014 .

[20]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[21]  Qi Li,et al.  Nonparametric Econometrics: Theory and Practice , 2006 .

[22]  José R. Dorronsoro,et al.  Group Fused Lasso , 2013, ICANN.

[23]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[24]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[25]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[26]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[27]  Min Chen,et al.  Joint conditional Gaussian graphical models with multiple sources of genomic data , 2013, Front. Genet..

[28]  Xiao-Tong Yuan,et al.  Partial Gaussian Graphical Model Estimation , 2012, IEEE Transactions on Information Theory.

[29]  Jieping Ye,et al.  Structural Graphical Lasso for Learning Mouse Brain Connectivity , 2015, KDD.

[30]  Larry A. Wasserman,et al.  Time varying undirected graphs , 2008, Machine Learning.

[31]  E. Levina,et al.  Joint estimation of multiple graphical models. , 2011, Biometrika.

[32]  Hongzhe Li,et al.  A SPARSE CONDITIONAL GAUSSIAN GRAPHICAL MODEL FOR ANALYSIS OF GENETICAL GENOMICS DATA. , 2011, The annals of applied statistics.

[33]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[34]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[35]  Hongzhe Li,et al.  Joint Estimation of Multiple High-dimensional Precision Matrices. , 2016, Statistica Sinica.

[36]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[37]  Lei Han,et al.  Overlapping Decomposition for Gaussian Graphical Modeling , 2015, IEEE Transactions on Knowledge and Data Engineering.

[38]  Peter Wonka,et al.  Fused Multiple Graphical Lasso , 2012, SIAM J. Optim..

[39]  Hongzhe Li,et al.  Adjusting for high-dimensional covariates in sparse precision matrix estimation by ℓ1-penalization , 2013, J. Multivar. Anal..

[40]  Jing Ma,et al.  Joint Structural Estimation of Multiple Graphical Models , 2016, J. Mach. Learn. Res..

[41]  Mladen Kolar,et al.  Inference for Sparse Conditional Precision Matrices , 2014, 1412.7638.

[42]  Han Liu,et al.  Joint estimation of multiple graphical models from high dimensional time series , 2013, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[43]  Rainer Breitling,et al.  Expression Quantitative Trait Loci Are Highly Sensitive to Cellular Differentiation State , 2009, PLoS genetics.

[44]  Adam J Rothman,et al.  Sparse Multivariate Regression With Covariance Estimation , 2010, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[45]  Yuan Ji,et al.  BAYESIAN SPARSE GRAPHICAL MODELS FOR CLASSIFICATION WITH APPLICATION TO PROTEIN EXPRESSION DATA. , 2014, The annals of applied statistics.

[46]  Martin J. Wainwright,et al.  Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$ -Constrained Quadratic Programming (Lasso) , 2009, IEEE Transactions on Information Theory.

[47]  Pradeep Ravikumar,et al.  Sparse inverse covariance matrix estimation using quadratic approximation , 2011, MLSLP.

[48]  Ming Yuan,et al.  High Dimensional Inverse Covariance Matrix Estimation via Linear Programming , 2010, J. Mach. Learn. Res..