Beyond Submodular Maximization

While there are well-developed tools for maximizing a submodular function subject to a matroid constraint, there is much less work on the corresponding supermodular maximization problems. We develop new techniques for attacking these problems inspired by the continuous greedy method applied to the multi-linear extension of a submodular function. We first adapt the continuous greedy algorithm to work for general twice-continuously differentiable functions. The performance of the adapted algorithm depends on a new smoothness parameter. If $F:[0,1]^n\rightarrow\mathbb{R}$ is one-sided $\sigma$-smooth, then the approximation factor only depends on $\sigma$. We apply the new algorithm to a broad class of quadratic supermodular functions arising in diversity maximization. This captures metric diversity maximization ($\sigma=2$) and negative-type diversity ($\sigma=4$). We also develop new methods (inspired by swap rounding and approximate integer decomposition) for rounding quadratics over a matroid polytope. Together with the adapted continuous greedy this leads to a $O(\sigma^{3/2})$-approximation. This is the best asymptotic approximation known for this class of quadratics and the evidence suggests that it may be tight. We then consider general (non-quadratic) functions. We give a broad parameterized family of monotone functions which include submodular functions and the just-discussed supermodular family of discrete quadratics. Such set functions are called $\gamma$-meta-submodular. We develop local search algorithms with approximation factors that depend only on $\gamma$. We show that the $\gamma$-meta-submodular families include well-known function classes including meta-submodular functions ($\gamma=0$), proportionally submodular ($\gamma=1$), and diversity functions based on negative-type distances or Jensen-Shannon divergence (both $\gamma=2$) and (semi-)metric diversity functions.

[1]  Sepehr Abbasi Zadeh,et al.  Max-Sum Diversification, Monotone Submodular Functions and Semi-metric Spaces , 2015, ArXiv.

[2]  Alexandros G. Dimakis,et al.  Streaming Weak Submodularity: Interpreting Neural Networks on the Fly , 2017, NIPS.

[3]  Friedrich Eisenbrand,et al.  Max-Sum Diversity Via Convex Programming , 2015, SoCG.

[4]  Jan Vondrák,et al.  Maximizing a Monotone Submodular Function Subject to a Matroid Constraint , 2011, SIAM J. Comput..

[5]  Yaron Singer,et al.  Maximization of Approximately Submodular Functions , 2016, NIPS.

[6]  Maxim Sviridenko,et al.  Pipage Rounding: A New Method of Constructing Algorithms with Proven Performance Guarantee , 2004, J. Comb. Optim..

[7]  Nicole Immorlica,et al.  A Unifying Hierarchy of Valuations with Complements and Substitutes , 2014, AAAI.

[8]  Jon M. Kleinberg,et al.  Segmentation problems , 2004, JACM.

[9]  Mark W. Schmidt,et al.  Distributed Maximization of "Submodular plus Diversity" Functions for Multi-label Feature Selection on Huge Datasets , 2019, AISTATS.

[10]  Mark Jerrum,et al.  Large Cliques Elude the Metropolis Process , 1992, Random Struct. Algorithms.

[11]  Joseph Naor,et al.  A Unified Continuous Greedy Algorithm for Submodular Maximization , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[12]  Pushmeet Kohli,et al.  P³ & Beyond: Move Making Algorithms for Solving Higher Order Functions , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Martial Hebert,et al.  Efficient Feature Group Sequencing for Anytime Linear Prediction , 2014, UAI.

[14]  Hui Lin,et al.  A Class of Submodular Functions for Document Summarization , 2011, ACL.

[15]  A. Borodin A Proportionally Submodular Functions , 2015 .

[16]  Morteza Zadimoghaddam,et al.  Scalable Feature Selection via Distributed Diversity Maximization , 2017, AAAI.

[17]  Jeff A. Bilmes,et al.  Submodularity beyond submodular energies: Coupling edges in graph cuts , 2011, CVPR 2011.

[18]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[19]  Barun Chandra,et al.  Facility Dispersion and Remote Subgraphs , 1995, SWAT.

[20]  Andreas Krause,et al.  Guarantees for Greedy Maximization of Non-submodular Functions with Applications , 2017, ICML.

[21]  Andreas Krause,et al.  Near-optimal Observation Selection using Submodular Functions , 2007, AAAI.

[22]  Moran Feldman,et al.  Building a Good Team: Secretary Problems and the Supermodular Degree , 2015, SODA.

[23]  Morteza Zadimoghaddam,et al.  Online Submodular Welfare Maximization: Greedy Beats 1/2 in Random Order , 2015, STOC.

[24]  Aditya Bhaskara,et al.  Linear Relaxations for Finding Diverse Elements in Metric Spaces , 2016, NIPS.

[25]  Jan Vondrák,et al.  Optimal approximation for the submodular welfare problem in the value oracle model , 2008, STOC.

[26]  Nicholas Roy,et al.  Contextual Sequence Prediction with Application to Control Library Optimization , 2013 .

[27]  Friedrich Eisenbrand,et al.  Local Search for Max-Sum Diversification , 2016, SODA.

[28]  Guy Kortsarz,et al.  On Choosing a Dense Subgraph (Extended Abstract) , 1993, FOCS 1993.

[29]  Allan Borodin,et al.  Weakly Submodular Functions , 2014, ArXiv.

[30]  Alexandros G. Dimakis,et al.  Scalable Greedy Feature Selection via Weak Submodularity , 2017, AISTATS.

[31]  Huy L. Nguyen,et al.  A New Framework for Distributed Submodular Maximization , 2015, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[32]  Lisa Fleischer,et al.  Submodular Approximation: Sampling-based Algorithms and Lower Bounds , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[33]  Moran Feldman,et al.  Constrained Monotone Function Maximization and the Supermodular Degree , 2014, APPROX-RANDOM.

[34]  Vahab S. Mirrokni,et al.  Diversity maximization under matroid constraints , 2013, KDD.

[35]  Amin Karbasi,et al.  Weakly Submodular Maximization Beyond Cardinality Constraints: Does Randomization Help Greedy? , 2017, ICML.

[36]  Éva Tardos,et al.  Maximizing the Spread of Influence through a Social Network , 2015, Theory Comput..

[37]  Alexander Schrijver,et al.  A Combinatorial Algorithm Minimizing Submodular Functions in Strongly Polynomial Time , 2000, J. Comb. Theory B.

[38]  Laurence A. Wolsey,et al.  Best Algorithms for Approximating the Maximum of a Submodular Set Function , 1978, Math. Oper. Res..

[39]  Uriel Feige,et al.  Welfare maximization and the supermodular degree , 2013, ITCS '13.

[40]  S. S. Ravi,et al.  Heuristic and Special Case Algorithms for Dispersion Problems , 1994, Oper. Res..

[41]  Sanjeev Arora,et al.  Inapproximabilty of Densest κ-Subgraph from Average Case Hardness , 2011 .

[42]  Stefanie Jegelka,et al.  Submodular meets Structured: Finding Diverse Subsets in Exponentially-Large Structured Item Sets , 2014, NIPS.

[43]  Marie-Pierre Jolly,et al.  Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images , 2001, ICCV.

[44]  Jeff A. Bilmes,et al.  Submodular feature selection for high-dimensional acoustic score spaces , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[45]  Jan Vondrák,et al.  Maximizing a Submodular Set Function Subject to a Matroid Constraint (Extended Abstract) , 2007, IPCO.

[46]  Matthew J. Streeter,et al.  An Online Algorithm for Maximizing Submodular Functions , 2008, NIPS.

[47]  Abhimanyu Das,et al.  Submodular meets Spectral: Greedy Algorithms for Subset Selection, Sparse Approximation and Dictionary Selection , 2011, ICML.