Parameter Learning for Loopy Markov Random Fields with Structural Support Vector Machines

Discriminative learners for functions with structured outputs have been successfully applied to sequence prediction, parsing, sequence alignment, and other problems where exact inference is tractable. However, these learners face challenges when inference procedures must use approximations, e.g., for multi-label classification, clustering, image segmentation, or loopy graphical models. In this paper, we explore methods for training structural SVMs on problems where exact inference is intractable. In particular, we consider pairwise fully connected Markov random fields, using multi-label classification as an example application. We show how to adapt loopy belief propagation, greedy search, and linear programming relaxations as (approximate) separation oracles in the structural SVM cutting-plane training algorithm. In addition to theoretical characterizations, we empirically evaluate and analyze the resulting algorithms on six multi-label classification datasets.