Guaranteed Non-convex Optimization: Submodular Maximization over Continuous Domains

Submodular continuous functions are a category of (generally) non-convex/non-concave functions with a wide spectrum of applications. We characterize these functions and demonstrate that they can be maximized efficiently with approximation guarantees. Specifically, i) We introduce the weak DR property that gives a unified characterization of submodularity for all set, integer-lattice and continuous functions; ii) for maximizing monotone DR-submodular continuous functions under general down-closed convex constraints, we propose a Frank-Wolfe variant with $(1-1/e)$ approximation guarantee, and sub-linear convergence rate; iii) for maximizing general non-monotone submodular continuous functions subject to box constraints, we propose a DoubleGreedy algorithm with $1/3$ approximation guarantee. Submodular continuous functions naturally find applications in various real-world settings, including influence and revenue maximization with continuous assignments, sensor energy management, multi-resolution data summarization, facility location, etc. Experimental results show that the proposed algorithms efficiently generate superior solutions compared to baseline algorithms.

[1]  Joseph Naor,et al.  A Tight Linear Time (1/2)-Approximation for Unconstrained Submodular Maximization , 2012, 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science.

[2]  Alexander J. Smola,et al.  Fast Stochastic Methods for Nonsmooth Nonconvex Optimization , 2016, ArXiv.

[3]  Martin Jaggi,et al.  Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization , 2013, ICML.

[4]  Dirk P. Kroese,et al.  Handbook of Monte Carlo Methods , 2011 .

[5]  Philip Wolfe,et al.  An algorithm for quadratic programming , 1956 .

[6]  Vasileios Hatzivassiloglou,et al.  Event-Based Extractive Summarization , 2004 .

[7]  Huan Li,et al.  Accelerated Proximal Gradient Methods for Nonconvex Programming , 2015, NIPS.

[8]  Jan Vondrák,et al.  Optimal approximation for the submodular welfare problem in the value oracle model , 2008, STOC.

[9]  Andreas Krause,et al.  Cost-effective outbreak detection in networks , 2007, KDD '07.

[10]  Ken-ichi Kawarabayashi,et al.  Lagrangian Decomposition Algorithm for Allocating Marketing Channels , 2015, AAAI.

[11]  Andreas Krause,et al.  Distributed Submodular Maximization: Identifying Representative Elements in Massive Data , 2013, NIPS.

[12]  Boris Polyak Gradient methods for the minimisation of functionals , 1963 .

[13]  Alexander Schrijver,et al.  A Combinatorial Algorithm Minimizing Submodular Functions in Strongly Polynomial Time , 2000, J. Comb. Theory B.

[14]  L. Wolsey Maximising Real-Valued Submodular Functions: Primal and Dual Heuristics for Location Problems , 1982, Math. Oper. Res..

[15]  Shahar Dobzinski,et al.  From query complexity to computational complexity , 2012, STOC '12.

[16]  Maryam Fazel,et al.  Designing smoothing functions for improved worst-case competitive ratio in online optimization , 2016, NIPS.

[17]  Masakazu Kojima,et al.  Exact Solutions of Some Nonconvex Quadratic Optimization Problems via SDP and SOCP Relaxations , 2003, Comput. Optim. Appl..

[18]  Zeyuan Allen Zhu,et al.  Nearly-Linear Time Positive LP Solver with Faster Convergence Rate , 2015, STOC.

[19]  Samuel Burer,et al.  Globally solving nonconvex quadratic programming problems via completely positive programming , 2011, Mathematical Programming Computation.

[20]  Britta Peis,et al.  Submodular Function Maximization on the Bounded Integer Lattice , 2015, WAOA.

[21]  Satoru Iwata,et al.  A combinatorial strongly polynomial algorithm for minimizing submodular functions , 2001, JACM.

[22]  Shai Shalev-Shwartz,et al.  On Graduated Optimization for Stochastic Non-Convex Problems , 2015, ICML.

[23]  Zeyuan Allen Zhu,et al.  Variance Reduction for Faster Non-Convex Optimization , 2016, ICML.

[24]  Andreas Krause,et al.  Submodular Dictionary Selection for Sparse Representation , 2010, ICML.

[25]  Maxim Sviridenko,et al.  Pipage Rounding: A New Method of Constructing Algorithms with Proven Performance Guarantee , 2004, J. Comb. Optim..

[26]  Yuichi Yoshida,et al.  Maximizing Submodular Functions with the Diminishing Return Property over the Integer Lattice , 2015, ArXiv.

[27]  Stanislav Zivny,et al.  Maximizing Bisubmodular and k-Submodular Functions , 2014, SODA.

[28]  Ken-ichi Kawarabayashi,et al.  Optimal Budget Allocation: Theoretical Guarantee and Efficient Algorithm , 2014, ICML.

[29]  Andreas Krause,et al.  Submodular Function Maximization , 2014, Tractability.

[30]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[31]  Yurii Nesterov,et al.  Cubic regularization of Newton method and its global performance , 2006, Math. Program..

[32]  Satoru Iwata,et al.  Submodular function minimization , 2007, Math. Program..

[33]  藤重 悟 Submodular functions and optimization , 1991 .

[34]  Anima Anandkumar,et al.  Tensor decompositions for learning latent variable models , 2012, J. Mach. Learn. Res..

[35]  Yuichi Yoshida,et al.  Maximizing monotone submodular functions over the integer lattice , 2015, IPCO.

[36]  László Lovász,et al.  Submodular functions and convexity , 1982, ISMP.

[37]  Francis Bach,et al.  Submodular functions: from discrete to continuous domains , 2015, Mathematical Programming.

[38]  Hui Lin,et al.  Multi-document Summarization via Budgeted Maximization of Submodular Functions , 2010, NAACL.

[39]  Vahab S. Mirrokni,et al.  Optimal marketing strategies over social networks , 2008, WWW.

[40]  Andreas Krause,et al.  From MAP to Marginals: Variational Inference in Bayesian Submodular Models , 2014, NIPS.

[41]  Andreas Krause,et al.  Near-optimal Nonmyopic Value of Information in Graphical Models , 2005, UAI.

[42]  Pushmeet Kohli,et al.  Tractability: Practical Approaches to Hard Problems , 2013 .

[43]  Anima Anandkumar,et al.  Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods , 2017 .

[44]  Hui Lin,et al.  A Class of Submodular Functions for Document Summarization , 2011, ACL.

[45]  Andreas Krause,et al.  Adaptive Submodularity: Theory and Applications in Active Learning and Stochastic Optimization , 2010, J. Artif. Intell. Res..

[46]  Wenruo Bai,et al.  Deep Submodular Functions , 2017, ArXiv.

[47]  Suvrit Sra,et al.  Scalable nonconvex inexact proximal splitting , 2012, NIPS.

[48]  Ben Taskar,et al.  Near-Optimal MAP Inference for Determinantal Point Processes , 2012, NIPS.

[49]  Jan Vondrák,et al.  Maximizing a Submodular Set Function Subject to a Matroid Constraint (Extended Abstract) , 2007, IPCO.

[50]  Abhimanyu Das,et al.  Submodular meets Spectral: Greedy Algorithms for Subset Selection, Sparse Approximation and Dictionary Selection , 2011, ICML.

[51]  Francis R. Bach,et al.  Structured sparsity-inducing norms through submodular functions , 2010, NIPS.

[52]  Martin Skutella,et al.  Convex quadratic and semidefinite programming relaxations in scheduling , 2001, JACM.

[53]  Donald M. Topkis,et al.  Minimizing a Submodular Function on a Lattice , 1978, Oper. Res..

[54]  Jan Vondrák,et al.  Maximizing a Monotone Submodular Function Subject to a Matroid Constraint , 2011, SIAM J. Comput..

[55]  Maxim Sviridenko,et al.  A note on maximizing a submodular set function subject to a knapsack constraint , 2004, Oper. Res. Lett..

[56]  Huy L. Nguyen,et al.  A Reduction for Optimizing Lattice Submodular Functions with Diminishing Returns , 2016, ArXiv.

[57]  Vahab S. Mirrokni,et al.  Maximizing Non-Monotone Submodular Functions , 2011, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[58]  Alexander J. Smola,et al.  Fast Incremental Method for Nonconvex Optimization , 2016, ArXiv.

[59]  Jeff A. Bilmes,et al.  On Bisubmodular Maximization , 2012, AISTATS.

[60]  Yuichi Yoshida,et al.  A Generalization of Submodular Cover via the Diminishing Return Property on the Integer Lattice , 2015, NIPS.