Solving Constrained Combinatorial Optimisation Problems via MAP Inference without High-Order Penalties

Solving constrained combinatorial optimization problems via MAP inference is often achieved by introducing extra potential functions for each constraint. This can result in very high order potentials, e.g. a 2-order objective with pairwise potentials and a quadratic constraint over all N variables would correspond to an unconstrained objective with an order-N potential. This limits the practicality of such an approach, since inference with high order potentials is tractable only for a few special classes of functions. We propose an approach which is able to solve constrained combinatorial problems using belief propagation without increasing the order. For example, in our scheme the 2-order problem above remains order 2 instead of order N . Experiments on applications ranging from foreground detection, image reconstruction, quadratic knapsack, and the M-best solutions problem demonstrate the effectiveness and efficiency of our method. Moreover, we show several situations in which our approach outperforms commercial solvers like CPLEX and others designed for specific constrained MAP inference problems. Introduction Maximum a posteriori (MAP) inference for graphical models can be used to solve unconstrained combinatorial optimization problems, or constrained problems by introducing extra potential functions for each constraint (Ravanbakhsh, Rabbany, and Greiner 2014; Ravanbakhsh and Greiner 2014; Frey and Dueck 2007; Bayati, Shah, and Sharma 2005; Werner 2008). The main limitation of this approach is that it often results in very high order potentials. Problems with pairwise potentials, for example, are very common, and adding a quadratic constraint (order 2) over N variables results in an objective function of order N . Optimizing over such high-order potentials is tractable only for a few special classes of functions (Tarlow, Givoni, and Zemel 2010; Potetz and Lee 2008; Komodakis and Paragios 2009; Mézard, Parisi, and Zecchina 2002; Aguiar et al. 2011), such as linear functions. Recently, Lim, Jung, and Kohli (2014) proposed cuttingplane based methods to handle constrained problems without Copyright c © 2017, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. introducing very high order potentials. However, their approaches require exact solutions of a series of unconstrained MAP inference problems, which is, in general, intractable. Thus their approaches are again only applicable to a particular class of potentials and constraints. To tackle general constrained combinatorial problems, our overall idea is to formulate the unconstrained combinatorial problem as a linear program (LP) with local marginal polytope constraints only (which corresponds to a classical MAP inference problem), and then add the “real” constraints from the original combinatorial problem to the existing LP to form a new LP. Duality of the new LP absorbs the “real” constraints naturally, and yields a convenient message passing procedure. The proposed algorithm is guaranteed to find feasible solutions for a quite general set of constraints. We apply our method to problems including foreground detection, image reconstruction, quadratic knapsack, and the M-best solutions problem, and show several situations in which it outperforms the commercial optimization solver CPLEX. We also test our method against more restrictive approaches including Aguiar et al. (2011) and Lim, Jung, and Kohli (2014) on the subsets of our applications to which they are applicable. Our method outperforms these methods in most cases even in settings that favor them. Preliminaries Here we consider factor graphical models with discrete variables. Denote the graph G = (V,C), where V is the set of nodes, and C is a collection of subsets of V. Each c ∈ C is called a cluster. If we associate one random variable xi with each node, and let x = [xi]i∈V, then it is often assumed that the joint distribution of x belongs to the exponential family p(x) = 1 Z exp [∑ c∈C θc(xc) ] , where xc denotes the vector [xi]i∈c. The real-valued function θc(xc) is known as a potential function. Without loss of generality we make the following assumption to simplify the derivation: Assumption 1. For convenience we assume that: (1) C includes every node, i.e., ∀i ∈ V, {i} ∈ C; (2) C is closed under intersection, i.e., ∀c1, c2 ∈ C, it is true that c1 ∩ c2 ∈ C. MAP and its LP relaxations The goal of MAP inference is to find the most likely assignment of values to the random Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17)