Sample Complexity of Joint Structure Learning

This paper considers the problem of jointly recovering the structures of two graphical models with unknown edge structures. It is assumed that both graphs have the same number of nodes and a known subset of nodes have identical structures in both graphs. The classes of Ising models and Gaussian models are considered. For Ising models, the objective is to recover the connectivity of both graphs under an approximate recovery criterion. For Gaussian models, the objectives of edge structure recovery and inverse covariance estimation are considered. Information-theoretic bounds on the sample complexity for bounded probability of error under the aforementioned criteria are established and compared with the corresponding bounds on the sample complexity for recovering the graphs independently.

[1]  Han Liu,et al.  Joint estimation of multiple graphical models from high dimensional time series , 2013, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[2]  Vincent Y. F. Tan,et al.  High-dimensional structure estimation in Ising models: Local separation criterion , 2011, 1107.1736.

[3]  J. Lafferty,et al.  High-dimensional Ising model selection using ℓ1-regularized logistic regression , 2010, 1010.0311.

[4]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[5]  Christine B Peterson,et al.  Bayesian Inference of Multiple Gaussian Graphical Models , 2015, Journal of the American Statistical Association.

[6]  José M. F. Moura,et al.  Necessary conditions for consistent set-based graphical model selection , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[7]  Adam J. Rothman,et al.  Sparse permutation invariant covariance estimation , 2008, 0801.4837.

[8]  Hongyu Zhao,et al.  Joint analysis of expression profiles from multiple cancers improves the identification of microRNA-gene interactions , 2013, Bioinform..

[9]  Martin J. Wainwright,et al.  Information-Theoretic Limits of Selecting Binary Graphical Models in High Dimensions , 2009, IEEE Transactions on Information Theory.

[10]  Martin J. Wainwright,et al.  Information-theoretic bounds on model selection for Gaussian Markov random fields , 2010, 2010 IEEE International Symposium on Information Theory.

[11]  M. West,et al.  Sparse graphical models for exploring gene expression data , 2004 .

[12]  Volkan Cevher,et al.  On the Difficulty of Selecting Ising Models With Approximate Recovery , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[13]  Patrick Danaher,et al.  The joint graphical lasso for inverse covariance estimation across multiple classes , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[14]  George Michailidis,et al.  ESTIMATING HETEROGENEOUS GRAPHICAL MODELS FOR DISCRETE DATA WITH AN APPLICATION TO ROLL CALL VOTING. , 2015, The annals of applied statistics.

[15]  Peter Wonka,et al.  Fused Multiple Graphical Lasso , 2012, SIAM J. Optim..

[16]  Ludovic Denoyer,et al.  Learning latent representations of nodes for classifying in heterogeneous social networks , 2014, WSDM.

[17]  Pascal Van Hentenryck,et al.  Graphical models for optimal power flow , 2016, Constraints.

[18]  E. Levina,et al.  Joint estimation of multiple graphical models. , 2011, Biometrika.

[19]  Vince D. Calhoun,et al.  Joint sparse canonical correlation analysis for detecting differential imaging genetics modules , 2016, Bioinform..

[20]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[21]  R. Nowak,et al.  Network tomography and the identification of shared infrastructure , 2002, Conference Record of the Thirty-Sixth Asilomar Conference on Signals, Systems and Computers, 2002..

[22]  Alexandros G. Dimakis,et al.  On the Information Theoretic Limits of Learning Ising Models , 2014, NIPS.

[23]  Steffen L. Lauritzen,et al.  Graphical models in R , 1996 .

[24]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[25]  Su-In Lee,et al.  Node-based learning of multiple Gaussian graphical models , 2013, J. Mach. Learn. Res..

[26]  Chee Sun Won,et al.  Unsupervised segmentation of noisy and textured images using Markov random fields , 1992, CVGIP Graph. Model. Image Process..