Approximate Inference in Continuous Determinantal Processes

Determinantal point processes (DPPs) are random point processes well-suited for modeling repulsion. In machine learning, the focus of DPP-based models has been on diverse subset selection from a discrete and finite base set. This discrete setting admits an efficient sampling algorithm based on the eigendecomposition of the defining kernel matrix. Recently, there has been growing interest in using DPPs defined on continuous spaces. While the discrete-DPP sampler extends formally to the continuous case, computationally, the steps required are not tractable in general. In this paper, we present two efficient DPP sampling schemes that apply to a wide range of kernel functions: one based on low rank approximations via Nystrom and random Fourier feature techniques and another based on Gibbs sampling. We demonstrate the utility of continuous DPPs in repulsive mixture modeling and synthesizing human poses spanning activity spaces.

[1]  Rong Jin,et al.  Nyström Method vs Random Fourier Features: A Theoretical and Empirical Comparison , 2012, NIPS.

[2]  Mari Myllymäki,et al.  Second‐order spatial analysis of epidermal nerve fibers , 2011, Statistics in medicine.

[3]  Ryan P. Adams,et al.  Priors for Diversity in Generative Latent Variable Models , 2012, NIPS.

[4]  J. Andrew Bagnell,et al.  Maximum margin planning , 2006, ICML.

[5]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[6]  Ben Taskar,et al.  Discovering Diverse and Salient Threads in Document Collections , 2012, EMNLP.

[7]  C. K. Liu,et al.  Learning physics-based motion style with nonlinear inverse optimization , 2005, SIGGRAPH 2005.

[8]  Ben Taskar,et al.  k-DPPs: Fixed-Size Determinantal Point Processes , 2011, ICML.

[9]  Y. Peres,et al.  Determinantal Processes and Independence , 2005, math/0503110.

[10]  João Roberto dos Santos,et al.  Markov point processes for modeling of spatial forest patterns in Amazonia derived from interferometric height , 2005 .

[11]  E. Rains,et al.  Eynard–Mehta Theorem, Schur Process, and their Pfaffian Analogs , 2004, math-ph/0409059.

[12]  Benjamin Recht,et al.  Random Features for Large-Scale Kernel Machines , 2007, NIPS.

[13]  Christian P. Robert,et al.  Monte Carlo Statistical Methods (Springer Texts in Statistics) , 2005 .

[14]  O. Macchi The coincidence approach to stochastic point processes , 1975, Advances in Applied Probability.

[15]  R. Bernstein,et al.  PARTITIONING OF SPACE IN COMMUNITIES OF ANTS , 1979 .

[16]  Ben Taskar,et al.  Structured Determinantal Point Processes , 2010, NIPS.

[17]  Daryl J. Daley,et al.  An Introduction to the Theory of Point Processes , 2013 .

[18]  Catherine A. Sugar,et al.  Finding the Number of Clusters in a Dataset , 2003 .

[19]  M. Stephens Dealing with label switching in mixture models , 2000 .

[20]  Ben Taskar,et al.  Nystrom Approximation for Large-Scale Determinantal Processes , 2013, AISTATS.

[21]  J. Schur,et al.  Über Potenzreihen, die im Innern des Einheitskreises beschränkt sind. , 1917 .

[22]  Junhui Wang Consistent selection of the number of clusters via crossvalidation , 2010 .

[23]  David B. Dunson,et al.  Repulsive Mixtures , 2012, NIPS.

[24]  Alex Kulesza,et al.  Markov Determinantal Point Processes , 2012, UAI.

[25]  Michael J. McCourt,et al.  Stable Evaluation of Gaussian Radial Basis Function Interpolants , 2012, SIAM J. Sci. Comput..

[26]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[27]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[28]  Ben Taskar,et al.  Determinantal Point Processes for Machine Learning , 2012, Found. Trends Mach. Learn..

[29]  Ege Holger Rubak,et al.  Statistical aspects of determinantal point processes , 2012 .

[30]  Matthias W. Seeger,et al.  Using the Nyström Method to Speed Up Kernel Machines , 2000, NIPS.

[31]  Robert E. Schapire,et al.  A Game-Theoretic Approach to Apprenticeship Learning , 2007, NIPS.

[32]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .