Constructing Composite Likelihoods in General Random Fields

We propose a simple estimator based on composite likelihoods for parameter learning in random field models. The estimator can be applied to all discrete graphical models such as Markov random fields and conditional random fields, including ones with higher-order energies. It is computationally efficient because it requires only inference over treestructured subgraphs of the original graph, and it is consistent, that is, it asymptotically gives the optimal parameter estimate in the model class. We verify these conceptual advantages in synthetic experiments and demonstrate the difficulties encountered by popular alternative estimation approaches.