An Improved Private Mechanism for Small Databases

We study the problem of answering a workload of linear queries \(\mathcal {Q}\), on a database of size at most \(n = o(|\mathcal {Q}|)\) drawn from a universe \(\mathcal {U}\) under the constraint of (approximate) differential privacy. Nikolov, Talwar, and Zhang [NTZ13] proposed an efficient mechanism that, for any given \(\mathcal {Q}\) and \(n\), answers the queries with average error that is at most a factor polynomial in \(\log |\mathcal {Q}|\) and \(\log |\mathcal {U}|\) worse than the best possible. Here we improve on this guarantee and give a mechanism whose competitiveness ratio is at most polynomial in \(\log n\) and \(\log |\mathcal {U}|\), and has no dependence on \(|\mathcal {Q}|\). Our mechanism is based on the projection mechanism of [NTZ13], but in place of an ad-hoc noise distribution, we use a distribution which is in a sense optimal for the projection mechanism, and analyze it using convex duality and the restricted invertibility principle.

[1]  J. Bourgain,et al.  Invertibility of ‘large’ submatrices with applications to the geometry of Banach spaces and harmonic analysis , 1987 .

[2]  Tim Roughgarden,et al.  Interactive privacy via the median mechanism , 2009, STOC '10.

[3]  A. Lewis.  The Convex Analysis of Unitarily Invariant Matrix Functions , 1995 .

[4]  Adam D. Smith,et al.  The price of privately releasing contingency tables and the spectra of random matrices with correlated rows , 2010, STOC '10.

[5]  Moni Naor,et al.  Differential privacy under continual observation , 2010, STOC '10.

[6]  Moni Naor,et al.  On the complexity of differentially private data release: efficient algorithms and hardness results , 2009, STOC '09.

[7]  Irit Dinur,et al.  Revealing information while preserving privacy , 2003, PODS.

[8]  Jonathan Ullman,et al.  Answering n{2+o(1)} counting queries with differential privacy is hard , 2012, STOC '13.

[9]  Aleksandar Nikolov,et al.  Approximating Hereditary Discrepancy via Small Width Ellipsoids , 2013, SODA.

[10]  Elaine Shi,et al.  Private and Continual Release of Statistics , 2010, ICALP.

[11]  Aleksandar Nikolov,et al.  Using Convex Relaxations for Efficiently and Privately Releasing Marginals , 2014, SoCG.

[12]  D. Spielman,et al.  An elementary proof of the restricted invertibility theorem , 2009, 0911.1114.

[13]  Michael L. Overton,et al.  Optimality conditions and duality theory for minimizing sums of the largest eigenvalues of symmetric matrices , 2015, Math. Program..

[14]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[15]  Aleksandar Nikolov,et al.  Optimal private halfspace counting via discrepancy , 2012, STOC '12.

[16]  Katrina Ligett,et al.  A Simple and Practical Algorithm for Differentially Private Data Release , 2010, NIPS.

[17]  Moni Naor,et al.  Our Data, Ourselves: Privacy Via Distributed Noise Generation , 2006, EUROCRYPT.

[18]  Adam D. Smith,et al.  Composition attacks and auxiliary information in data privacy , 2008, KDD.

[19]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[20]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[21]  K. Fan.  On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations I. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Aaron Roth,et al.  Privately Releasing Conjunctions and the Statistical Query Barrier , 2013, SIAM J. Comput..

[23]  Cynthia Dwork,et al.  The price of privacy and the limits of LP decoding , 2007, STOC '07.

[24]  Aleksandar Nikolov,et al.  The geometry of differential privacy: the sparse and approximate cases , 2012, STOC '13.

[25]  Aaron Roth,et al.  Iterative Constructions and Private Data Release , 2011, TCC.

[26]  Li Zhang,et al.  Nearly optimal minimax estimator for high-dimensional sparse linear regression , 2012, 1206.6536.

[27]  Aaron Roth,et al.  A learning theory approach to noninteractive database privacy , 2011, JACM.

[28]  Aleksandar Nikolov.  Randomized Rounding for the Largest Simplex Problem , 2015, STOC.

[29]  K. Fan.  On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations: II. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Martin Grötschel,et al.  The ellipsoid method and its consequences in combinatorial optimization , 1981, Comb..

[31]  Guy N. Rothblum,et al.  Boosting and Differential Privacy , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[32]  Cynthia Dwork,et al.  Privacy-Preserving Datamining on Vertically Partitioned Databases , 2004, CRYPTO.

[33]  Guy N. Rothblum,et al.  A Multiplicative Weights Mechanism for Privacy-Preserving Data Analysis , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[34]  Alexander Schrijver,et al.  Combinatorial optimization. Polyhedra and efficiency. , 2003 .