Submodular Point Processes

We introduce a class of discrete point processes that we call the Submodular Point Processes (SPPs). These processes are characterized via a submodular (or supermodular) function, and naturally model notions of information, coverage and diversity, as well as cooperation. Unlike Log-submodular and Log-supermodular distributions (Log-SPPs) such as determinantal point processes (DPPs), SPPs are themselves submodular (or supermodular). In this paper, we analyze the computational complexity of probabilistic inference in SPPs. We show that computing the partition function for SPPs (and Log-SPPs), requires exponential complexity in the worst case, and also provide algorithms which approximate SPPs up to polynomial factors. Moreover, for several subclasses of interesting submodular functions that occur in applications, we show how we can provide efficient closed form expressions for the partition functions, and thereby marginals and conditional distributions. Finally, we argue how SPPs complement existing Log-SPP distributions, and are a natural model for several applications.

[1]  H. Ishikawa.  Higher-order clique reduction in binary graph cut , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Rishabh K. Iyer,et al.  Fast Semidifferential-based Submodular Function Optimization , 2013, ICML.

[3]  Deeparnab Chakrabarty,et al.  Knapsack Problems , 2008 .

[4]  J. Bilmes,et al.  Submodularity in natural language processing: algorithms and applications , 2012 .

[5]  Mohit Singh,et al.  On the Approximation of Submodular Functions , 2013, ArXiv.

[6]  Hui Lin,et al.  Optimal Selection of Limited Vocabulary Speech Corpora , 2011, INTERSPEECH.

[7]  Jeff A. Bilmes,et al.  Ratio semi-definite classifiers , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[8]  Andreas Krause,et al.  From MAP to Marginals: Variational Inference in Bayesian Submodular Models , 2014, NIPS.

[9]  Rishabh K. Iyer,et al.  Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints , 2013, NIPS.

[10]  Hui Lin,et al.  A Class of Submodular Functions for Document Summarization , 2011, ACL.

[11]  Jeff A. Bilmes,et al.  Submodular subset selection for large-scale speech training data , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Vahab S. Mirrokni,et al.  Approximating submodular functions everywhere , 2009, SODA.

[13]  Ben Taskar,et al.  Determinantal Point Processes for Machine Learning , 2012, Found. Trends Mach. Learn..

[14]  Steven M. Seitz,et al.  Scene Summarization for Online Image Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[15]  Eric Vigoda,et al.  A Deterministic Polynomial-Time Approximation Scheme for Counting Knapsack Solutions , 2010, SIAM J. Comput..

[16]  Hui Lin,et al.  Learning Mixtures of Submodular Shells with Application to Document Summarization , 2012, UAI.

[17]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[18]  Rishabh K. Iyer,et al.  Curvature and Optimal Algorithms for Learning and Minimizing Submodular Functions , 2013, NIPS.

[19]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Jeff A. Bilmes,et al.  Submodularity beyond submodular energies: Coupling edges in graph cuts , 2011, CVPR 2011.

[21]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Dafna Shahaf,et al.  Turning down the noise in the blogosphere , 2009, KDD.

[23]  Jingrui He,et al.  GenDeR: A Generic Diversified Ranking Algorithm , 2012, NIPS.

[24]  Rishabh K. Iyer,et al.  Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with Applications , 2012, UAI.

[25]  O. Macchi.  The coincidence approach to stochastic point processes , 1975, Advances in Applied Probability.

[26]  Hui Lin,et al.  Multi-document Summarization via Budgeted Maximization of Submodular Functions , 2010, NAACL.

[27]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[28]  Mark Jerrum,et al.  Polynomial-Time Approximation Algorithms for the Ising Model , 1990, SIAM J. Comput..

[29]  Hui Lin,et al.  Graph-based submodular selection for extractive summarization , 2009, 2009 IEEE Workshop on Automatic Speech Recognition & Understanding.

[30]  Andreas Krause,et al.  Learning Fourier Sparse Set Functions , 2012, AISTATS.