Integral Privacy for Sampling from Mollifier Densities with Approximation Guarantees

$\varepsilon$-differential privacy is a leading protection setting, focused by design on individual privacy. Many applications, such as in the medical / pharmaceutical domains, would rather posit privacy at a group level, furthermore of unknown size, a setting in which classical budget scaling tricks typically cannot guarantee non-trivial privacy levels. We call this privacy setting integral privacy. In this paper, we study a major problem of machine learning and statistics with related applications in domains cited above that have recently met with substantial press: sampling. Our formal contribution is twofolds: we provide a general theory for sampling to be integrally private, and we show how to achieve integral privacy with guarantees on the approximation of the true (non-private) density. Our theory introduces $\varepsilon$-mollifiers, subsets of densities whose sampling is guaranteed to be integrally private. Guaranteed approximation bounds of the true density are obtained via the boosting theory as it was originally formulated: we learn sufficient statistics in an $\varepsilon$-mollifier of exponential families using classifiers, which brings guaranteed approximation and convergence rates that degrade gracefully with the privacy budget, under weak assumptions. Approximation guarantees cover the mode capture problem. Experimental results against private kernel density estimation and private GANs displays the quality of our results, in particular for high privacy regimes.