Computing Differential Privacy Guarantees for Heterogeneous Compositions Using FFT

The recently proposed Fast Fourier Transform (FFT)-based accountant for evaluating (ε, δ)-differential privacy guarantees using the privacy loss distribution formalism has been shown to give tighter bounds than commonly used methods such as Rényi accountants when applied to compositions of homogeneous mechanisms. This approach is also applicable to certain discrete mechanisms that cannot be analysed with Rényi accountants. We extend this approach to compositions of heterogeneous mechanisms. We carry out a full error analysis that allows choosing the parameters of the algorithm such that a desired accuracy is obtained.

[1]  Ilya Mironov,et al.  Rényi Differential Privacy , 2017, 2017 IEEE 30th Computer Security Foundations Symposium (CSF).

[2]  Aaron Roth,et al.  Gaussian differential privacy , 2019, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[3]  Gilles Barthe,et al.  Beyond Differential Privacy: Composition Theorems and Relational Logic for f-divergences between Probabilistic Programs , 2013, ICALP.

[4]  David M. Sommer,et al.  Privacy Loss Classes: The Central Limit Theorem in Differential Privacy , 2019, IACR Cryptol. ePrint Arch..

[5]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[6]  Ilya Mironov,et al.  On significance of the least significant bits for differential privacy , 2012, CCS.

[7]  Yu-Xiang Wang,et al.  Poission Subsampled Rényi Differential Privacy , 2019, ICML.

[8]  Oliver Kosut,et al.  A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergences , 2020, 2020 IEEE International Symposium on Information Theory (ISIT).

[9]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[10]  Arnold Neumaier,et al.  Introduction to Numerical Analysis , 2001 .

[11]  J. Tukey,et al.  An algorithm for the machine calculation of complex Fourier series , 1965 .

[12]  Thomas G. Stockham,et al.  High-speed convolution and correlation , 1966, AFIPS '66 (Spring).

[13]  Thomas Steinke,et al.  The Discrete Gaussian for Differential Privacy , 2020, NeurIPS.

[14]  Salil P. Vadhan,et al.  The Complexity of Computing the Optimal Composition of Differential Privacy , 2015, IACR Cryptol. ePrint Arch..

[15]  Sergio Verdú,et al.  $f$ -Divergence Inequalities , 2015, IEEE Transactions on Information Theory.

[16]  Yu-Xiang Wang,et al.  Subsampled Rényi Differential Privacy and Analytical Moments Accountant , 2018, AISTATS.

[17]  T. O’Neil.  Geometric Measure Theory , 2002 .

[18]  Gilles Barthe,et al.  Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences , 2018, NeurIPS.

[19]  Jinshuo Dong,et al.  Deep Learning with Gaussian Differential Privacy , 2020, Harvard data science review.

[20]  Sanjiv Kumar,et al.  cpSGD: Communication-efficient and differentially-private distributed SGD , 2018, NeurIPS.

[21]  G. A. Young,et al.  High‐dimensional Statistics: A Non‐asymptotic Viewpoint, Martin J.Wainwright, Cambridge University Press, 2019, xvii 552 pages, £57.99, hardback ISBN: 978‐1‐1084‐9802‐9 , 2020, International Statistical Review.

[22]  Li Zhang,et al.  Rényi Differential Privacy of the Sampled Gaussian Mechanism , 2019, ArXiv.

[23]  Antti Honkela,et al.  Tight Differential Privacy for Discrete-Valued Mechanisms and for the Subsampled Gaussian Mechanism Using FFT , 2020, AISTATS.

[24]  Richard Nock,et al.  Advances and Open Problems in Federated Learning , 2021, Found. Trends Mach. Learn..

[25]  Ananda Theertha Suresh,et al.  Shuffled Model of Federated Learning: Privacy, Communication and Accuracy Trade-offs , 2020, ArXiv.

[26]  Esfandiar Mohammadi,et al.  Tight on Budget?: Tight Bounds for r-Fold Approximate Differential Privacy , 2018, CCS.

[27]  A. Honkela,et al.  Computing Tight Differential Privacy Guarantees Using FFT , 2019, AISTATS.

[28]  Antti Honkela,et al.  Tight Approximate Differential Privacy for Discrete-Valued Mechanisms Using FFT , 2020, ArXiv.

[29]  Edwin Hewitt,et al.  Real and Abstract Analysis: A Modern Treatment of the Theory of Functions of a Real Variable , 1965 .