Non-monotone Submodular Maximization with Nearly Optimal Adaptivity and Query Complexity

Submodular maximization is a general optimization problem with a wide range of applications in machine learning (e.g., active learning, clustering, and feature selection). In large-scale optimization, the parallel running time of an algorithm is governed by its adaptivity, which measures the number of sequential rounds needed if the algorithm can execute polynomially-many independent oracle queries in parallel. While low adaptivity is ideal, it is not sufficient for an algorithm to be efficient in practice---there are many applications of distributed submodular optimization where the number of function evaluations becomes prohibitively expensive. Motivated by these applications, we study the adaptivity and query complexity of submodular maximization. In this paper, we give the first constant-factor approximation algorithm for maximizing a non-monotone submodular function subject to a cardinality constraint $k$ that runs in $O(\log(n))$ adaptive rounds and makes $O(n \log(k))$ oracle queries in expectation. In our empirical study, we use three real-world applications to compare our algorithm with several benchmarks for non-monotone submodular maximization. The results demonstrate that our algorithm finds competitive solutions using significantly fewer rounds and queries.

[1]  Brendan J. Frey,et al.  Non-metric affinity propagation for unsupervised image categorization , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[2]  Sergei Vassilvitskii,et al.  Fast greedy algorithms in mapreduce and streaming , 2013, SPAA.

[3]  Thorsten Joachims,et al.  Temporal corpus summarization using submodular word coverage , 2012, CIKM '12.

[4]  Gary L. Miller,et al.  Graph Sketching against Adaptive Adversaries Applied to the Minimum Degree Algorithm , 2018, 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS).

[5]  Silvio Lattanzi,et al.  Filtering: a method for solving graph problems in MapReduce , 2011, SPAA '11.

[6]  Andreas Krause,et al.  Lazier Than Lazy Greedy , 2014, AAAI.

[7]  Huy L. Nguyen,et al.  Submodular maximization with matroid and packing constraints in parallel , 2018, STOC.

[8]  Jan Vondrák,et al.  Submodular maximization by simulated annealing , 2010, SODA '11.

[9]  Jan Vondrák,et al.  Submodular Optimization in the MapReduce Model , 2018, SOSA.

[10]  Ola Svensson,et al.  Beyond 1/2-Approximation for Submodular Maximization on Massive Data Streams , 2018, ICML.

[11]  Alexandros G. Dimakis,et al.  Restricted Strong Convexity Implies Weak Submodularity , 2016, The Annals of Statistics.

[12]  Vahab S. Mirrokni,et al.  Maximizing Nonmonotone Submodular Functions under Matroid or Knapsack Constraints , 2009, SIAM J. Discret. Math..

[13]  Arpit Agarwal,et al.  Stochastic Submodular Cover with Limited Adaptivity , 2019, SODA.

[14]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[15]  U. Feige,et al.  Maximizing Non-monotone Submodular Functions , 2011 .

[16]  Kent Quanrud,et al.  Submodular Function Maximization in Parallel via the Multilinear Relaxation , 2019, SODA.

[17]  Andreas Krause,et al.  Deletion-Robust Submodular Maximization: Data Summarization with "the Right to be Forgotten" , 2017, ICML.

[18]  Abhimanyu Das,et al.  Algorithms for subset selection in linear regression , 2008, STOC.

[19]  Ameet Talwalkar,et al.  Foundations of Machine Learning , 2012, Adaptive computation and machine learning.

[20]  Vahab S. Mirrokni,et al.  Optimal marketing strategies over social networks , 2008, WWW.

[21]  Eric Balkanski,et al.  An Exponential Speedup in Parallel Running Time for Submodular Maximization without Loss in Approximation , 2018, SODA.

[22]  Andreas Krause,et al.  Distributed Submodular Maximization: Identifying Representative Elements in Massive Data , 2013, NIPS.

[23]  Morteza Zadimoghaddam,et al.  Submodular Maximization with Nearly Optimal Approximation, Adaptivity and Query Complexity , 2018, SODA.

[24]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[25]  Robert Tibshirani,et al.  Spectral Regularization Algorithms for Learning Large Incomplete Matrices , 2010, J. Mach. Learn. Res..

[26]  Baharan Mirzasoleiman,et al.  Fast Constrained Submodular Maximization: Personalized Data Summarization , 2016, ICML.

[27]  Eric Balkanski,et al.  Non-monotone Submodular Maximization in Exponentially Fewer Iterations , 2018, NeurIPS.

[28]  Alexandros G. Dimakis,et al.  Scalable Greedy Feature Selection via Weak Submodularity , 2017, AISTATS.

[29]  Huy L. Nguyen,et al.  A New Framework for Distributed Submodular Maximization , 2015, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[30]  Joseph Naor,et al.  Submodular Maximization with Cardinality Constraints , 2014, SODA.

[31]  Huy L. Nguyen,et al.  Submodular Maximization with Nearly-optimal Approximation and Adaptivity in Nearly-linear Time , 2018, SODA.

[32]  Yaron Singer,et al.  Fast Parallel Algorithms for Feature Selection , 2019, ArXiv.

[33]  Carlos Guestrin,et al.  Beyond keyword search: discovering relevant scientific literature , 2011, KDD.

[34]  Andreas Krause,et al.  Streaming submodular maximization: massive data summarization on the fly , 2014, KDD.

[35]  Rishabh K. Iyer,et al.  Learning Mixtures of Submodular Functions for Image Collection Summarization , 2014, NIPS.

[36]  Huy L. Nguyen,et al.  The Power of Randomization: Distributed Submodular Maximization on Massive Datasets , 2015, ICML.

[37]  Eric Balkanski,et al.  The adaptive complexity of maximizing a submodular function , 2018, STOC.

[38]  Jure Leskovec,et al.  {SNAP Datasets}: {Stanford} Large Network Dataset Collection , 2014 .

[39]  Huy L. Nguyen,et al.  A Parallel Double Greedy Algorithm for Submodular Maximization , 2018, ArXiv.

[40]  A. An,et al.  Scalable Deletion-Robust Submodular Maximization: Data Summarization with Privacy and Fairness Constraints , 2018 .

[41]  Kent Quanrud,et al.  Parallelizing greedy for submodular set function maximization in matroids and beyond , 2018, STOC.

[42]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[43]  Steven M. Seitz,et al.  Scene Summarization for Online Image Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[44]  Roy Schwartz,et al.  Comparing Apples and Oranges: Query Trade-off in Submodular Maximization , 2017, Math. Oper. Res..

[45]  Morteza Zadimoghaddam,et al.  Randomized Composable Core-sets for Distributed Submodular Maximization , 2015, STOC.

[46]  Amin Karbasi,et al.  Unconstrained submodular maximization with constant adaptive complexity , 2019, STOC.

[47]  Niv Buchbinder,et al.  Constrained Submodular Maximization via a Non-symmetric Technique , 2016, Math. Oper. Res..