Pain-Free Random Differential Privacy with Sensitivity Sampling

Popular approaches to differential privacy, such as the Laplace and exponential mechanisms, calibrate randomised smoothing through global sensitivity of the target non-private function. Bounding such sensitivity is often a prohibitively complex analytic calculation. As an alternative, we propose a straightforward sampler for estimating sensitivity of non-private mechanisms. Since our sensitivity estimates hold with high probability, any mechanism that would be $(\epsilon,\delta)$-differentially private under bounded global sensitivity automatically achieves $(\epsilon,\delta,\gamma)$-random differential privacy (Hall et al., 2012), without any target-specific calculations required. We demonstrate on worked example learners how our usable approach adopts a naturally-relaxed privacy guarantee, while achieving more accurate releases even for non-private functions that are black-box computer programs.

[1]  Daniel Kifer,et al.  Private Convex Empirical Risk Minimization and High-dimensional Regression , 2012, COLT 2012.

[2]  Frank McSherry,et al.  Privacy integrated queries: an extensible platform for privacy-preserving data analysis , 2009, SIGMOD Conference.

[3]  Christos Dimitrakakis,et al.  Differential Privacy for Bayesian Inference through Posterior Sampling , 2017, J. Mach. Learn. Res..

[4]  Catuscia Palamidessi,et al.  Differential Privacy for Relational Algebra: Improving the Sensitivity Bounds via Constraint Systems , 2012, QAPL.

[5]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[6]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[7]  Christos Dimitrakakis,et al.  Robust and Private Bayesian Inference , 2013, ALT.

[8]  Kunal Talwar,et al.  Mechanism Design via Differential Privacy , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[9]  Ratul Mahajan,et al.  Differentially-private network trace analysis , 2010, SIGCOMM '10.

[10]  Hiroshi Nakagawa,et al.  Differential Privacy without Sensitivity , 2016, NIPS.

[11]  Andreas Haeberlen,et al.  Linear dependent types for differential privacy , 2013, POPL.

[12]  Andreas Haeberlen,et al.  Differential Privacy Under Fire , 2011, USENIX Security Symposium.

[13]  Darakhshan J. Mir Differentially-private learning and information theory , 2012, EDBT-ICDT '12.

[14]  Elaine Shi,et al.  GUPT: privacy preserving data analysis made easy , 2012, SIGMOD Conference.

[15]  Adam D. Smith,et al.  Differentially Private Feature Selection via Stability Arguments, and the Robustness of the Lasso , 2013, COLT.

[16]  Ling Huang,et al.  Learning in a Large Function Space: Privacy-Preserving Mechanisms for SVM Learning , 2009, J. Priv. Confidentiality.

[17]  Benjamin I. P. Rubinstein,et al.  The Bernstein Mechanism: Function Release under Differential Privacy , 2017, AAAI.

[18]  Eli Upfal,et al.  Mining Frequent Itemsets through Progressive Sampling with Rademacher Averages , 2015, KDD.

[19]  Anand D. Sarwate,et al.  Differentially Private Empirical Risk Minimization , 2009, J. Mach. Learn. Res..

[20]  Alexander J. Smola,et al.  Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlo , 2015, ICML.

[21]  P. Massart The Tight Constant in the Dvoretzky-Kiefer-Wolfowitz Inequality , 1990 .

[22]  Ioannis Chatzigeorgiou,et al.  Bounds on the Lambert Function and Their Application to the Outage Analysis of User Cooperation , 2013, IEEE Communications Letters.

[23]  Gilles Barthe,et al.  Programming language techniques for differential privacy , 2016, SIGL.

[24]  Ilya Mironov,et al.  Differentially private recommender systems: building privacy into the net , 2009, KDD.

[25]  Sofya Raskhodnikova,et al.  Smooth sensitivity and sampling in private data analysis , 2007, STOC '07.

[26]  Larry A. Wasserman,et al.  Random Differential Privacy , 2011, J. Priv. Confidentiality.

[27]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[28]  Benjamin C. Pierce,et al.  Distance makes the types grow stronger: a calculus for differential privacy , 2010, ICFP '10.

[29]  Stephen T. Joy The Differential Privacy of Bayesian Inference , 2015 .

[30]  Vitaly Shmatikov,et al.  Airavat: Security and Privacy for MapReduce , 2010, NSDI.