Local conditional and marginal approach to parameter estimation in discrete graphical models

Abstract Discrete graphical models are an essential tool in the identification of the relationship between variables in complex high-dimensional problems. When the number of variables p is large, computing the maximum likelihood estimate (henceforth abbreviated MLE) of the parameter is difficult. A popular approach is to estimate the composite MLE (abbreviated MCLE) rather than the MLE, i.e., the value of the parameter that maximizes the product of local conditional or local marginal likelihoods, centered around each vertex v of the graph underlying the model. The purpose of this paper is to first show that, when all the neighbors of v are linked to other nodes in the graph, the estimates obtained through local conditional and marginal likelihoods are identical. Thus the two MCLE are usually very close. Second, we study the asymptotic properties of the composite MLE obtained by averaging of the estimates from the local conditional likelihoods: this is done under the double asymptotic regime when both p and N go to infinity.

[1]  A. Hero,et al.  Distributed Covariance Estimation in , 2012 .

[2]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[3]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[4]  Stephen E. Fienberg,et al.  Maximum likelihood estimation in log-linear models , 2011, 1104.3618.

[5]  T. Speed,et al.  Additive and Multiplicative Models and Interactions , 1983 .

[6]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[7]  M. Frydenberg.  Marginalization and Collapsibility in Graphical Interaction Models , 1990 .

[8]  C. Geyer.  Markov Chain Monte Carlo Maximum Likelihood , 1991 .

[9]  Alfred O. Hero,et al.  Marginal Likelihoods for Distributed Parameter Estimation of Gaussian Graphical Models , 2014, IEEE Transactions on Signal Processing.

[10]  Misha Denil,et al.  Distributed Parameter Estimation in Probabilistic Graphical Models , 2014, NIPS.

[11]  Bin Yu,et al.  High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence , 2008, 0811.3628.

[12]  H. Massam,et al.  Approximating faces of marginal polytopes in discrete hierarchical models , 2016, The Annals of Statistics.

[13]  Alfred O. Hero,et al.  Distributed Covariance Estimation in Gaussian Graphical Models , 2010, IEEE Transactions on Signal Processing.

[14]  Alfred O. Hero,et al.  Distributed Learning of Gaussian Graphical Models via Marginal Likelihoods , 2013, AISTATS.

[15]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[16]  J. Lafferty,et al.  High-dimensional Ising model selection using ℓ1-regularized logistic regression , 2010, 1010.0311.

[17]  Carlo Gaetan,et al.  Composite likelihood methods for space-time data , 2006 .

[18]  Qiang Liu,et al.  Distributed Parameter Estimation via Pseudo-likelihood , 2012, ICML.

[19]  Helene Massam,et al.  Bayes factors and the geometry of discrete hierarchical loglinear models , 2011, 1103.5381.