Average-Case Averages: Private Algorithms for Smooth Sensitivity and Mean Estimation

The simplest and most widely applied method for guaranteeing differential privacy is to add instance-independent noise to a statistic of interest that is scaled to its global sensitivity. However, global sensitivity is a worst-case notion that is often too conservative for realized dataset instances. We provide methods for scaling noise in an instance-dependent way and demonstrate that they provide greater accuracy under average-case distributional assumptions. Specifically, we consider the basic problem of privately estimating the mean of a real distribution from i.i.d.~samples. The standard empirical mean estimator can have arbitrarily-high global sensitivity. We propose the trimmed mean estimator, which interpolates between the mean and the median, as a way of attaining much lower sensitivity on average while losing very little in terms of statistical accuracy. To privately estimate the trimmed mean, we revisit the smooth sensitivity framework of Nissim, Raskhodnikova, and Smith (STOC 2007), which provides a framework for using instance-dependent sensitivity. We propose three new additive noise distributions which provide concentrated differential privacy when scaled to smooth sensitivity. We provide theoretical and experimental evidence showing that our noise distributions compare favorably to others in the literature, in particular, when applied to the mean estimation problem.

[1]  Jun Sakuma,et al.  Differentially Private Analysis of Outliers , 2015, ECML/PKDD.

[2]  Vishesh Karwa,et al.  Finite Sample Differentially Private Confidence Intervals , 2017, ITCS.

[3]  Guy N. Rothblum,et al.  Concentrated Differential Privacy , 2016, ArXiv.

[4]  Thomas Steinke,et al.  Calibrating Noise to Variance in Adaptive Data Analysis , 2017, COLT.

[5]  Anna M. Ritz,et al.  Differentially Private ANOVA Testing , 2017, 2018 1st International Conference on Data Intelligence and Security (ICDIS).

[6]  John Duchi,et al.  Lower Bounds for Locally Private Estimation via Communication Complexity , 2019, COLT.

[7]  Thomas Steinke,et al.  Composable and versatile privacy via truncated CDP , 2018, STOC.

[8]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[9]  Moni Naor,et al.  Our Data, Ourselves: Privacy Via Distributed Noise Generation , 2006, EUROCRYPT.

[10]  Ilya Mironov,et al.  Rényi Differential Privacy , 2017, 2017 IEEE 30th Computer Security Foundations Symposium (CSF).

[11]  Marco Gaboardi,et al.  Locally Private Mean Estimation: Z-test and Tight Confidence Intervals , 2018, AISTATS.

[12]  Adam D. Smith,et al.  The structure of optimal private tests for simple hypotheses , 2018, STOC.

[13]  Jonathan Ullman,et al.  Efficiently Estimating Erdos-Renyi Graphs with Node Differential Privacy , 2019, NeurIPS.

[14]  Thomas Steinke,et al.  Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds , 2016, TCC.

[15]  Sofya Raskhodnikova,et al.  Smooth sensitivity and sampling in private data analysis , 2007, STOC '07.

[16]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[17]  Alon Gonen,et al.  Smooth Sensitivity Based Approach for Differentially Private Principal Component Analysis , 2017, ArXiv.

[18]  Xintao Wu,et al.  Preserving Differential Privacy in Degree-Correlation based Graph Generation , 2013, Trans. Data Priv..

[19]  Victor-Emmanuel Brunel,et al.  Differentially private sub-Gaussian location estimators , 2019, 1906.11923.

[20]  Úlfar Erlingsson,et al.  Scalable Private Learning with PATE , 2018, ICLR.

[21]  Adam D. Smith,et al.  Efficient, Differentially Private Point Estimators , 2008, ArXiv.

[22]  Janardhan Kulkarni,et al.  Locally Private Gaussian Estimation , 2018, NeurIPS.

[23]  Md Zahidul Islam,et al.  Ieee Transactions on Knowledge and Data Engineering 1 Differentially Private Random Decision Forests Using Smooth Sensitivity , 2022 .

[24]  Jonathan Ullman,et al.  Private Identity Testing for High-Dimensional Distributions , 2019, NeurIPS.

[25]  Raef Bassily,et al.  Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds , 2014, 1405.7085.

[26]  Sofya Raskhodnikova,et al.  Analyzing Graphs with Node Differential Privacy , 2013, TCC.

[27]  Adam Groce,et al.  Differentially Private Nonparametric Hypothesis Testing , 2019, CCS.

[28]  Sofya Raskhodnikova,et al.  Private analysis of graph structure , 2011, Proc. VLDB Endow..

[29]  Robert Serfling,et al.  Asymptotic Relative Efficiency in Estimation , 2011, International Encyclopedia of Statistical Science.

[30]  Ryan M. Rogers,et al.  Differentially Private Chi-Squared Hypothesis Testing: Goodness of Fit and Independence Testing , 2016, ICML 2016.

[31]  Yue Wang,et al.  Differentially Private Hypothesis Testing, Revisited , 2015, ArXiv.

[32]  Cynthia Dwork,et al.  Differential privacy and robust statistics , 2009, STOC '09.