Approximate Recovery Of Ising Models with Side Information

This paper considers the problem of recovering the edge structures of two partially identical graphs in the class of Ising models. It is assumed that both graphs have the same number of nodes and a known subset of nodes have identical structures in both graphs. Therefore, inferring the structure of one graph can provide the side information that could be leveraged for inference related to the other graph. The objective is to recover the connectivity of both graphs under an approximate recovery criterion. The degree- and edge-bounded subclass of Ising models is considered and necessary conditions (information-theoretic) and sufficient conditions for the sample complexity to achieve a bounded probability of error are established. Furthermore, the scaling behavior of the sample complexity is analyzed in different regimes and specific regimes are identified for which the necessary and sufficient conditions coincide, thus, establishing the optimal sample complexity.

[1]  Han Liu,et al.  Joint estimation of multiple graphical models from high dimensional time series , 2013, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[2]  Tyler VanderWeele,et al.  Causality, 2nd edn , 2011 .

[3]  Volkan Cevher,et al.  On the Difficulty of Selecting Ising Models With Approximate Recovery , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[4]  Pascal Van Hentenryck,et al.  Graphical models for optimal power flow , 2016, Constraints.

[5]  E. Levina,et al.  Joint estimation of multiple graphical models. , 2011, Biometrika.

[6]  Peter Wonka,et al.  Fused Multiple Graphical Lasso , 2012, SIAM J. Optim..

[7]  Hongyu Zhao,et al.  Joint analysis of expression profiles from multiple cancers improves the identification of microRNA-gene interactions , 2013, Bioinform..

[8]  Martin J. Wainwright,et al.  Information-theoretic bounds on model selection for Gaussian Markov random fields , 2010, 2010 IEEE International Symposium on Information Theory.

[9]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[10]  José M. F. Moura,et al.  Necessary conditions for consistent set-based graphical model selection , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[11]  Christine B Peterson,et al.  Bayesian Inference of Multiple Gaussian Graphical Models , 2015, Journal of the American Statistical Association.

[12]  Su-In Lee,et al.  Node-based learning of multiple Gaussian graphical models , 2013, J. Mach. Learn. Res..

[13]  J. Lafferty,et al.  High-dimensional Ising model selection using ℓ1-regularized logistic regression , 2010, 1010.0311.

[14]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[15]  Ali Tajer,et al.  Sample Complexity of Joint Structure Learning , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Ali Tajer,et al.  Structure Learning of Similar Ising Models: Information-theoretic Bounds , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[17]  Vince D. Calhoun,et al.  Joint sparse canonical correlation analysis for detecting differential imaging genetics modules , 2016, Bioinform..

[18]  Martin J. Wainwright,et al.  Information-Theoretic Limits of Selecting Binary Graphical Models in High Dimensions , 2009, IEEE Transactions on Information Theory.

[19]  George Michailidis,et al.  ESTIMATING HETEROGENEOUS GRAPHICAL MODELS FOR DISCRETE DATA WITH AN APPLICATION TO ROLL CALL VOTING. , 2015, The annals of applied statistics.

[20]  Alexandros G. Dimakis,et al.  On the Information Theoretic Limits of Learning Ising Models , 2014, NIPS.

[21]  Ali Tajer,et al.  Structure Learning with Side Information: Sample Complexity , 2019, NeurIPS.

[22]  Ludovic Denoyer,et al.  Learning latent representations of nodes for classifying in heterogeneous social networks , 2014, WSDM.

[23]  R. Nowak,et al.  Network tomography and the identification of shared infrastructure , 2002, Conference Record of the Thirty-Sixth Asilomar Conference on Signals, Systems and Computers, 2002..

[24]  Chee Sun Won,et al.  Unsupervised segmentation of noisy and textured images using Markov random fields , 1992, CVGIP Graph. Model. Image Process..

[25]  M. West,et al.  Sparse graphical models for exploring gene expression data , 2004 .

[26]  Adam J. Rothman,et al.  Sparse permutation invariant covariance estimation , 2008, 0801.4837.

[27]  Patrick Danaher,et al.  The joint graphical lasso for inverse covariance estimation across multiple classes , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[28]  Michael I. Jordan.  Graphical Models , 2003 .