Learning Determinantal Point Processes with Moments and Cycles

Determinantal Point Processes (DPPs) are a family of probabilistic models that have a repulsive behavior, and lend themselves naturally to many tasks in machine learning where returning a diverse set of objects is important. While there are fast algorithms for sampling, marginalization and conditioning, much less is known about learning the parameters of a DPP. Our contribution is twofold: (i) we establish the optimal sample complexity achievable in this problem and show that it is governed by a natural parameter, which we call the \emph{cycle sparsity}; (ii) we propose a provably fast combinatorial algorithm that implements the method of moments efficiently and achieves optimal sample complexity. Finally, we give experimental results that confirm our theoretical findings.

[1]  Mohit Singh,et al.  Maximizing determinants under partition constraints , 2016, STOC.

[2]  Suvrit Sra,et al.  Fast Sampling for Strongly Rayleigh Measures with Application to Determinantal Point Processes , 2016, ArXiv.

[3]  Hui Lin,et al.  Learning Mixtures of Submodular Shells with Application to Document Summarization , 2012, UAI.

[4]  Aleksandar Nikolov Randomized Rounding for the Largest Simplex Problem , 2015, STOC.

[5]  Alex Kulesza,et al.  Diversifying Sparsity Using Variational Determinantal Point Processes , 2014, ArXiv.

[6]  Edoardo Amaldi,et al.  Efficient Deterministic Algorithms for Finding a Minimum Cycle Basis in Undirected Graphs , 2010, IPCO.

[7]  Ben Taskar,et al.  Learning the Parameters of Determinantal Point Process Kernels , 2014, ICML.

[8]  Ben Taskar,et al.  k-DPPs: Fixed-Size Determinantal Point Processes , 2011, ICML.

[9]  Joseph Douglas Horton,et al.  A Polynomial-Time Algorithm to Find the Shortest Cycle Basis of a Graph , 1987, SIAM J. Comput..

[10]  Ben Taskar,et al.  Learning Determinantal Point Processes , 2011, UAI.

[11]  Ben Taskar,et al.  Determinantal Point Processes for Machine Learning , 2012, Found. Trends Mach. Learn..

[12]  Edward Y. Chang,et al.  Tweet Timeline Generation with Determinantal Point Processes , 2016, AAAI.

[13]  Jasper Snoek,et al.  A Determinantal Point Process Latent Variable Model for Inhibition in Neural Spiking Data , 2013, NIPS.

[14]  Malik Magdon-Ismail,et al.  On selecting a maximum volume sub-matrix of a matrix and related problems , 2009, Theor. Comput. Sci..

[15]  Amin Karbasi,et al.  Fast Mixing for Discrete Point Processes , 2015, COLT.

[16]  Donghoon Lee,et al.  Individualness and Determinantal Point Processes for Pedestrian Detection , 2016, ECCV.

[17]  Suvrit Sra,et al.  Fast DPP Sampling for Nystrom with Application to Kernel Methods , 2016, ICML.

[18]  John C. Urschel,et al.  Maximum likelihood estimation of determinantal point processes , 2017, 1701.06501.

[19]  F. Dyson Statistical Theory of the Energy Levels of Complex Systems. I , 1962 .

[20]  Ben Taskar,et al.  Expectation-Maximization for Learning Determinantal Point Processes , 2014, NIPS.

[21]  Ben Taskar,et al.  An efficient algorithm for the symmetric principal minor assignment problem , 2015 .

[22]  David Maxwell Chickering,et al.  On Finding a Cycle Basis with a Shortest Maximal Cycle , 1995, Inf. Process. Lett..

[23]  Robert E. Tarjan,et al.  Algorithmic Aspects of Vertex Elimination on Graphs , 1976, SIAM J. Comput..

[24]  O. Macchi The coincidence approach to stochastic point processes , 1975, Advances in Applied Probability.

[25]  E. Rains,et al.  Eynard–Mehta Theorem, Schur Process, and their Pfaffian Analogs , 2004, math-ph/0409059.

[26]  Suvrit Sra,et al.  Fixed-point algorithms for learning determinantal point processes , 2015, ICML.

[27]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[28]  Luis Rademacher,et al.  Efficient Volume Sampling for Row/Column Subset Selection , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[29]  Zhijian Ou,et al.  Scalable Discovery of Audio Fingerprint Motifs in Broadcast Streams With Determinantal Point Process Based Motif Clustering , 2016, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[30]  Friedrich Eisenbrand,et al.  On largest volume simplices and sub-determinants , 2014, SODA.