Fast Optimization of Boolean Quadratic Functions via Iterative Submodular Approximation and Max-flow

We consider the NP–hard combinatorial optimization problem of minimizing arbitrary quadratic forms over the {0, 1 } (Boolean) lattice. While polynomial-time approximation algorithms do exist for such problems, they suffer from the practical drawback of being computationally involved – often a side effect of being agnostic to the combinatorial structure inherent in the problem. In this paper, we propose a computationally lightweight approximation alternative which specifically exploits the combinatorial structure of the problem. The key result underlying our approach is that any Boolean quadratic function can be expressed as a difference of quadratic sub-modular functions, which enables us to construct and iteratively min-imize a sequence of global submodular upper bounds on the cost function. This entails solving a quadratic submodular function min-imization problem at each step, which can be efficiently accomplished via the seminal Max-Flow algorithm. Overall, our algorithm performs iterative approximation by solving a sequence of maximum-flow problems. The merits of using this approach are illustrated via simulations which indicate the very favorable performance of our algorithm.

[1]  Satoru Iwata,et al.  A combinatorial, strongly polynomial-time algorithm for minimizing submodular functions , 2000, STOC '00.

[2]  D. R. Fulkerson,et al.  Flows in Networks. , 1964 .

[3]  Andrew V. Goldberg,et al.  A new approach to the maximum flow problem , 1986, STOC '86.

[4]  Amir Beck,et al.  A sequential parametric convex approximation method with applications to nonconvex truss topology design problems , 2010, J. Glob. Optim..

[5]  George N. Karystinos,et al.  Efficient computation of the binary vector that maximizes a rank-deficient quadratic form , 2010, IEEE Trans. Inf. Theory.

[6]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[7]  Rishabh K. Iyer,et al.  Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with Applications , 2012, UAI.

[8]  Alexander Schrijver,et al.  A Combinatorial Algorithm Minimizing Submodular Functions in Strongly Polynomial Time , 2000, J. Comb. Theory B.

[9]  Stephen P. Boyd,et al.  A semidefinite programming method for integer convex quadratic minimization , 2015, Optimization Letters.

[10]  Gordon P. Wright,et al.  Technical Note - A General Inner Approximation Algorithm for Nonconvex Mathematical Programs , 1978, Oper. Res..

[11]  Francis R. Bach,et al.  Learning with Submodular Functions: A Convex Optimization Perspective , 2011, Found. Trends Mach. Learn..

[12]  Jeff A. Bilmes,et al.  A Submodular-supermodular Procedure with Applications to Discriminative Structure Learning , 2005, UAI.

[13]  Shuzhong Zhang,et al.  Quadratic maximization and semidefinite relaxation , 2000, Math. Program..

[14]  Satoru Iwata,et al.  A combinatorial strongly polynomial algorithm for minimizing submodular functions , 2001, JACM.

[15]  Martin Grötschel,et al.  The ellipsoid method and its consequences in combinatorial optimization , 1981, Comb..

[16]  Francisco Facchinei,et al.  Parallel and Distributed Methods for Constrained Nonconvex Optimization—Part I: Theory , 2016, IEEE Transactions on Signal Processing.

[17]  László Lovász,et al.  Submodular functions and convexity , 1982, ISMP.

[18]  S. Fujishige,et al.  A Submodular Function Minimization Algorithm Based on the Minimum-Norm Base ⁄ , 2009 .

[19]  Zhi-Quan Luo,et al.  Quasi-maximum-likelihood multiuser detection using semi-definite relaxation with application to synchronous CDMA , 2002, IEEE Trans. Signal Process..

[20]  Yin Tat Lee,et al.  A Faster Cutting Plane Method and its Implications for Combinatorial and Convex Optimization , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[21]  Y. Nesterov Semidefinite relaxation and nonconvex quadratic optimization , 1998 .

[22]  Zhi-Quan Luo,et al.  Semidefinite Relaxation of Quadratic Optimization Problems , 2010, IEEE Signal Processing Magazine.

[23]  Jack Edmonds,et al.  Submodular Functions, Matroids, and Certain Polyhedra , 2001, Combinatorial Optimization.

[24]  Francisco Facchinei,et al.  Parallel and Distributed Methods for Constrained Nonconvex Optimization-Part II: Applications in Communications and Machine Learning , 2017, IEEE Transactions on Signal Processing.

[25]  Zhi-Quan Luo,et al.  SDP relaxation of homogeneous quadratic optimization: Approximation bounds and applications , 2009 .

[26]  James B. Orlin,et al.  A faster strongly polynomial time algorithm for submodular function minimization , 2007, Math. Program..

[27]  Jacques Stern,et al.  The hardness of approximate optima in lattices, codes, and systems of linear equations , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[28]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Satoru Fujishige,et al.  Submodular functions and optimization , 1991 .