Algorithms for Optimizing the Ratio of Submodular Functions

We investigate a new optimization problem involving minimizing the Ratio of two Submodular (RS) functions. We argue that this problem occurs naturally in several real world applications. We then show the connection between this problem and several related problems including minimizing the difference between submodular functions (Iyer & Bilmes, 2012b; Narasimhan & Bilmes, 2005), and to submodular optimization subject to submodular constraints (Iyer & Bilmes, 2013). We show that RS optimization can be solved with bounded approximation factors. We also provide a hardness bound and show that our tightest algorithm matches the lower bound up to a log factor. Finally, we empirically demonstrate the performance and good scalability properties of our algorithms.

[1]  Rishabh K. Iyer,et al.  Mixed Robust/Average Submodular Partitioning: Fast Algorithms, Guarantees, and Applications , 2015, NIPS.

[2]  Jeff A. Bilmes,et al.  Submodularity beyond submodular energies: Coupling edges in graph cuts , 2011, CVPR 2011.

[3]  Rishabh K. Iyer,et al.  Submodularity in Data Subset Selection and Active Learning , 2015, ICML.

[4]  Rishabh K. Iyer,et al.  Curvature and Optimal Algorithms for Learning and Minimizing Submodular Functions , 2013, NIPS.

[5]  Hui Lin,et al.  Optimal Selection of Limited Vocabulary Speech Corpora , 2011, INTERSPEECH.

[6]  Rishabh K. Iyer,et al.  Submodular-Bregman and the Lovász-Bregman Divergences with Applications , 2012, NIPS.

[7]  Jeff A. Bilmes,et al.  A Submodular-supermodular Procedure with Applications to Discriminative Structure Learning , 2005, UAI.

[8]  Rishabh K. Iyer,et al.  Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints , 2013, NIPS.

[9]  Vahab S. Mirrokni,et al.  Approximating submodular functions everywhere , 2009, SODA.

[10]  Rishabh K. Iyer,et al.  Fast Multi-stage Submodular Maximization , 2014, ICML.

[11]  Andreas Krause,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008, J. Mach. Learn. Res..

[12]  Jan Vondr.  Submodularity and Curvature: The Optimal Algorithm , 2010 .

[13]  Michel Minoux,et al.  Accelerated greedy algorithms for maximizing submodular set functions , 1978 .

[14]  Jeff A. Bilmes,et al.  Using Document Summarization Techniques for Speech Data Subset Selection , 2013, NAACL.

[15]  Alper Atamtürk,et al.  The submodular knapsack polytope , 2009, Discret. Optim..

[16]  Gérard Cornuéjols,et al.  Submodular set functions, matroids and the greedy algorithm: Tight worst-case bounds and some generalizations of the Rado-Edmonds theorem , 1984, Discret. Appl. Math..

[17]  Lisa Fleischer,et al.  Submodular Approximation: Sampling-based Algorithms and Lower Bounds , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[18]  Rishabh K. Iyer,et al.  Fast Semidifferential-based Submodular Function Optimization , 2013, ICML.

[19]  Laurence A. Wolsey,et al.  An analysis of the greedy algorithm for the submodular set covering problem , 1982, Comb..

[20]  Washio Takashi,et al.  Prismatic algorithm for discrete D.C. programming problem , 2011 .

[21]  Jeff A. Bilmes,et al.  Submodular feature selection for high-dimensional acoustic score spaces , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[22]  Yoshio Okamoto,et al.  Submodular fractional programming for balanced clustering , 2011, Pattern Recognit. Lett..

[23]  U. Feige,et al.  Maximizing Non-monotone Submodular Functions , 2011 .

[24]  Satoru Fujishige,et al.  Submodular functions and optimization , 1991 .

[25]  Jeff A. Bilmes,et al.  Local Search for Balanced Submodular Clusterings , 2007, IJCAI.

[26]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  Rishabh K. Iyer,et al.  Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with Applications , 2012, UAI.

[28]  László Lovász,et al.  Submodular functions and convexity , 1982, ISMP.