Submodular Function Maximization

Submodularity is a property of set functions with deep theoretical consequences and far– reaching applications. At first glance it appears very similar to concavity, in other ways it resembles convexity. It appears in a wide variety of applications: in Computer Science it has recently been identified and utilized in domains such as viral marketing (Kempe et al., 2003), information gathering (Krause and Guestrin, 2007), image segmentation (Boykov and Jolly, 2001; Kohli et al., 2009; Jegelka and Bilmes, 2011a), document summarization (Lin and Bilmes, 2011), and speeding up satisfiability solvers (Streeter and Golovin, 2008). In this survey we will introduce submodularity and some of its generalizations, illustrate how it arises in various applications, and discuss algorithms for optimizing submodular functions. Our emphasis here is on maximization; there are many important results and applications related to minimizing submodular functions that we do not cover.

[1]  Y. Freund,et al.  Adaptive game playing using multiplicative weights , 1999 .

[2]  Satoru Fujishige,et al.  Polymatroidal Dependence Structure of a Set of Random Variables , 1978, Inf. Control..

[3]  Amin Saberi,et al.  Stochastic Submodular Maximization , 2008, WINE.

[4]  G. Nemhauser,et al.  Maximizing Submodular Set Functions: Formulations and Analysis of Algorithms* , 1981 .

[5]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[6]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[7]  Andreas Krause,et al.  Online distributed sensor selection , 2010, IPSN '10.

[8]  Maxim Sviridenko,et al.  Pipage Rounding: A New Method of Constructing Algorithms with Proven Performance Guarantee , 2004, J. Comb. Optim..

[9]  Jeff A. Bilmes,et al.  Simultaneous Learning and Covering with Adversarial Noise , 2011, ICML.

[10]  Andreas S. Schulz,et al.  Revisiting the Greedy Approach to Submodular Set Function Maximization , 2007 .

[11]  Andreas Krause,et al.  Efficient Sensor Placement Optimization for Securing Large Water Distribution Networks , 2008 .

[12]  Jeff A. Bilmes,et al.  Submodularity beyond submodular energies: Coupling edges in graph cuts , 2011, CVPR 2011.

[13]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[14]  Andreas Krause,et al.  Near-optimal Nonmyopic Value of Information in Graphical Models , 2005, UAI.

[15]  Pieter Abbeel,et al.  Grasping and Fixturing as Submodular Coverage Problems , 2011, ISRR.

[16]  Sanjoy Dasgupta,et al.  Analysis of a greedy active learning strategy , 2004, NIPS.

[17]  H. B. McMahan,et al.  Robust Submodular Observation Selection , 2008 .

[18]  Jan Vondrák,et al.  Optimal approximation for the submodular welfare problem in the value oracle model , 2008, STOC.

[19]  Stephen F. Smith,et al.  Combining Multiple Heuristics Online , 2007, AAAI.

[20]  Andreas Krause,et al.  Cost-effective outbreak detection in networks , 2007, KDD '07.

[21]  Laurence A. Wolsey,et al.  Best Algorithms for Approximating the Maximum of a Submodular Set Function , 1978, Math. Oper. Res..

[22]  Jeff A. Bilmes,et al.  PAC-learning Bounded Tree-width Graphical Models , 2004, UAI.

[23]  Jan Vondrák,et al.  Maximizing a Submodular Set Function Subject to a Matroid Constraint (Extended Abstract) , 2007, IPCO.

[24]  Matthew J. Streeter,et al.  An Online Algorithm for Maximizing Submodular Functions , 2008, NIPS.

[25]  Gustav Bergmann.  Zur Axiomatik der Elementargeometrie , 1929 .

[26]  C. Scott,et al.  Modified Group Generalized Binary Search with Near-Optimal Performance Guarantees , 2010 .

[27]  U. Feige,et al.  Maximizing Non-monotone Submodular Functions , 2011 .

[28]  Satoru Fujishige,et al.  Submodular functions and optimization , 1991 .

[29]  Alexander Schrijver,et al.  Combinatorial optimization. Polyhedra and efficiency. , 2003 .

[30]  Andreas Krause,et al.  Online Learning of Assignments , 2009, NIPS.

[31]  Stephen F. Smith,et al.  Restart Schedules for Ensembles of Problem Instances , 2007, AAAI.

[32]  Jack Edmonds,et al.  Submodular Functions, Matroids, and Certain Polyhedra , 2001, Combinatorial Optimization.

[33]  Pushmeet Kohli,et al.  P³ & Beyond: Move Making Algorithms for Solving Higher Order Functions , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Morteza Zadimoghaddam,et al.  Submodular secretary problem and extensions , 2013, TALG.

[35]  Andreas Krause,et al.  Near-Optimal Bayesian Active Learning with Noisy Observations , 2010, NIPS.

[36]  Hui Lin,et al.  A Class of Submodular Functions for Document Summarization , 2011, ACL.

[37]  A. M. FRIEZE.  A cost function property for plant location problems , 1974, Math. Program..

[38]  Andreas Krause,et al.  Adaptive Submodularity: Theory and Applications in Active Learning and Stochastic Optimization , 2010, J. Artif. Intell. Res..

[39]  Jeff A. Bilmes,et al.  Interactive Submodular Set Cover , 2010, ICML.

[40]  Peter Auer,et al.  The Nonstochastic Multiarmed Bandit Problem , 2002, SIAM J. Comput..

[41]  Andreas Krause,et al.  Adaptive Submodular Optimization under Matroid Constraints , 2011, ArXiv.

[42]  Jan Vondrák,et al.  Submodular maximization by simulated annealing , 2010, SODA '11.

[43]  Andreas Krause,et al.  Near Optimal Bayesian Active Learning for Decision Making , 2014, AISTATS.

[44]  Filip Radlinski,et al.  Learning diverse rankings with multi-armed bandits , 2008, ICML '08.

[45]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[46]  Hui Lin,et al.  Multi-document Summarization via Budgeted Maximization of Submodular Functions , 2010, NAACL.

[47]  Andreas Krause,et al.  Simultaneous placement and scheduling of sensors , 2009, 2009 International Conference on Information Processing in Sensor Networks.

[48]  Laurence A. Wolsey,et al.  An analysis of the greedy algorithm for the submodular set covering problem , 1982, Comb..

[49]  Andreas Krause,et al.  Nonmyopic Adaptive Informative Path Planning for Multiple Robots , 2009, IJCAI.

[50]  Andreas Krause,et al.  SFO: A Toolbox for Submodular Function Optimization , 2010, J. Mach. Learn. Res..

[51]  G. A. Tijssen,et al.  The Data-Correcting Algorithm for the Minimization of Supermodular Functions , 1999 .

[52]  Chandra Chekuri,et al.  A recursive greedy algorithm for walks in directed graphs , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[53]  Elad Hazan,et al.  Online submodular minimization , 2009, J. Mach. Learn. Res..

[54]  Zhanxing Zhu,et al.  Neural Information Processing Systems (NIPS) , 2015 .

[55]  Teresa M. Przytycka,et al.  On an Optimal Split Tree Problem , 1999, WADS.

[56]  Aaron Roth,et al.  Constrained Non-monotone Submodular Maximization: Offline and Secretary Algorithms , 2010, WINE.

[57]  Vahab Mirrokni,et al.  Maximizing Non-Monotone Submodular Functions , 2007, FOCS 2007.

[58]  Éva Tardos,et al.  Maximizing the Spread of Influence through a Social Network , 2015, Theory Comput..

[59]  Jeff A. Bilmes,et al.  Q-Clustering , 2005, NIPS.

[60]  Jeff A. Bilmes,et al.  Online Submodular Minimization for Combinatorial Structures , 2011, ICML.

[61]  Andreas Krause,et al.  Randomized Sensing in Adversarial Environments , 2011, IJCAI.

[62]  Andreas Krause,et al.  Efficient Planning of Informative Paths for Multiple Robots , 2006, IJCAI.

[63]  Marie-Pierre Jolly,et al.  Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images , 2001, ICCV.

[64]  J. Vondrák,et al.  Submodular Function Maximization via the Multilinear Relaxation and Contention Resolution Schemes , 2014 .

[65]  Andreas Krause,et al.  Efficient Minimization of Decomposable Submodular Functions , 2010, NIPS.

[66]  Gruia Calinescu,et al.  The Polymatroid Steiner Problems , 2005, J. Comb. Optim..

[67]  Jennifer Widom,et al.  The Pipelined Set Cover Problem , 2005, ICDT.

[68]  Jennifer Widom,et al.  Adaptive ordering of pipelined stream filters , 2004, SIGMOD '04.

[69]  Jeff A. Bilmes,et al.  Submodularity Cuts and Applications , 2009, NIPS.

[70]  Andreas Krause,et al.  Near-optimal Observation Selection using Submodular Functions , 2007, AAAI.

[71]  Andreas Krause,et al.  Dynamic Resource Allocation in Conservation Planning , 2011, AAAI.

[72]  Andreas Krause,et al.  A Utility-Theoretic Approach to Privacy and Personalization , 2008, AAAI.

[73]  Michel Minoux,et al.  Accelerated greedy algorithms for maximizing submodular set functions , 1978 .

[74]  László Lovász,et al.  Submodular functions and convexity , 1982, ISMP.

[75]  Jan Vondrák,et al.  Maximizing a Monotone Submodular Function Subject to a Matroid Constraint , 2011, SIAM J. Comput..

[76]  C. Guestrin,et al.  Near-optimal sensor placements: maximizing information while minimizing communication cost , 2006, 2006 5th International Conference on Information Processing in Sensor Networks.

[77]  Maxim Sviridenko,et al.  A note on maximizing a submodular set function subject to a knapsack constraint , 2004, Oper. Res. Lett..

[78]  Matthew J. Streeter,et al.  Combining Multiple Constraint Solvers : Results on the CPAI ’ 06 Competition Data , 2007 .

[79]  Andreas Krause,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008, J. Mach. Learn. Res..

[80]  Jan Vondr.  Submodularity and Curvature: The Optimal Algorithm , 2010 .

[81]  Hadas Shachnai,et al.  Maximizing submodular set functions subject to multiple linear constraints , 2009, SODA.

[82]  G. Birkhoff,et al.  On the combination of subalgebras , 1933, Mathematical Proceedings of the Cambridge Philosophical Society.

[83]  Hao Yang,et al.  Near-optimal algorithms for shared filter evaluation in data stream systems , 2008, SIGMOD Conference.