On Multiplicative Weight Updates for Concave and Submodular Function Maximization

We develop a continuous-time framework based on multiplicative weight updates to approximately solve continuous optimization problems. The framework allows for a simple and modular analysis for a variety of problems involving convex constraints and concave or submodular objective functions. The continuous-time framework avoids the cumbersome technical details that are typically necessary in actual algorithms. We also show that the continuous-time algorithms can be converted into implementable algorithms via a straightforward discretization process. Using our framework and additional ideas we obtain significantly faster algorithms compared to previously known algorithms to maximize the multilinear relaxation of a monotone or non-monotone submodular set function subject to linear packing constraints.

[1]  Jan Vondrák,et al.  Maximizing a Monotone Submodular Function Subject to a Matroid Constraint , 2011, SIAM J. Comput..

[2]  Jan Vondrák,et al.  Dependent Randomized Rounding via Exchange Properties of Combinatorial Structures , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[3]  Chandra Chekuri,et al.  Submodular function maximization via the multilinear relaxation and contention resolution schemes , 2011, STOC '11.

[4]  Sanjeev Arora,et al.  The Multiplicative Weights Update Method: a Meta-Algorithm and Applications , 2012, Theory Comput..

[5]  Leonid Khachiyan,et al.  Approximate Max-Min Resource Sharing for Structured Concave Optimization , 2000, SIAM J. Optim..

[6]  Leonid Khachiyan,et al.  Fast Approximation Schemes for Convex Programs with Many Blocks and Coupling Constraints , 1994, SIAM J. Optim..

[7]  Hadas Shachnai,et al.  Approximations for Monotone and Nonmonotone Submodular Maximization with Knapsack Constraints , 2013, Math. Oper. Res..

[8]  Yossi Azar,et al.  Efficient Submodular Function Maximization under Linear Packing Constraints , 2012, ICALP.

[9]  Satyen Kale.  Efficient algorithms using the multiplicative weights update method , 2007 .

[10]  Daniel Bienstock,et al.  Potential Function Methods for Approximately Solving Linear Programming Problems: Theory and Practice , 2002 .

[11]  Sergei Vassilvitskii,et al.  Fast greedy algorithms in mapreduce and streaming , 2013, SPAA.

[12]  Jan Vondrák,et al.  Optimal approximation for the submodular welfare problem in the value oracle model , 2008, STOC.

[13]  Noam Nisan,et al.  A parallel approximation algorithm for positive linear programming , 1993, STOC.

[14]  Oded Regev,et al.  Simulating Quantum Correlations with Finite Communication , 2007, FOCS.

[15]  Christian Borgs,et al.  Maximizing Social Influence in Nearly Optimal Time , 2012, SODA.

[16]  Joseph Naor,et al.  A Unified Continuous Greedy Algorithm for Submodular Maximization , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[17]  Sylvain Sorin,et al.  Exponential weight algorithm in continuous time , 2008, Math. Program..

[18]  Lisa Fleischer,et al.  Approximating Fractional Multicommodity Flow Independent of the Number of Commodities , 2000, SIAM J. Discret. Math..

[19]  Panayotis Mertikopoulos,et al.  A continuous-time approach to online optimization , 2014, Journal of Dynamics & Games.

[20]  Klaus Jansen,et al.  Approximation algorithms for general packing problems and their application to the multicast congestion problem , 2008, Math. Program..

[21]  Jochen Könemann,et al.  Faster and simpler algorithms for multicommodity flow and other fractional packing problems , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[22]  Éva Tardos,et al.  Fast Approximation Algorithms for Fractional Packing and Covering Problems , 1995, Math. Oper. Res..

[23]  JansenKlaus,et al.  Faster and simpler approximation algorithms for mixed packing and covering problems , 2007 .

[24]  Rishabh K. Iyer,et al.  Fast Semidifferential-based Submodular Function Optimization , 2013, ICML.

[25]  Naveen Garg,et al.  Fractional Covering with Upper Bounds on the Variables: Solving LPs with Negative Entries , 2004, ESA.

[26]  Ben Taskar,et al.  Near-Optimal MAP Inference for Determinantal Point Processes , 2012, NIPS.

[27]  Sanjeev Arora,et al.  A combinatorial, primal-dual approach to semidefinite programs , 2007, STOC '07.

[28]  Jan Vondrák,et al.  Maximizing a Submodular Set Function Subject to a Matroid Constraint (Extended Abstract) , 2007, IPCO.

[29]  Sanjeev Arora,et al.  Fast algorithms for approximate semidefinite programming using the multiplicative weights update method , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[30]  Ravi Kumar,et al.  Max-cover in map-reduce , 2010, WWW '10.

[31]  Andreas Krause,et al.  Streaming submodular maximization: massive data summarization on the fly , 2014, KDD.

[32]  R. Khandekar.  Lagrangian relaxation based algorithms for convex programming problems , 2004 .

[33]  Neal E. Young,et al.  Randomized rounding without solving the linear program , 1995, SODA '95.

[34]  Neal E. Young,et al.  Sequential and parallel algorithms for mixed packing and covering , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[35]  Jochen Könemann,et al.  Fast Combinatorial Algorithms for Packing and Covering Problems , 2000 .

[36]  Neal E. Young.  Nearly Linear-Time Approximation Schemes for Mixed Packing/Covering and Facility-Location Linear Programs , 2014, ArXiv.

[37]  Jan Vondrák,et al.  Fast algorithms for maximizing submodular functions , 2014, SODA.

[38]  Vahab S. Mirrokni,et al.  Maximizing Nonmonotone Submodular Functions under Matroid or Knapsack Constraints , 2009, SIAM J. Discret. Math..

[39]  Aravind Srinivasan,et al.  Solving Packing Integer Programs via Randomized Rounding with Alterations , 2012, Theory Comput..