Local Differential Privacy Is Equivalent to Contraction of $E_\gamma$-Divergence

We investigate the local differential privacy (LDP) guarantees of a randomized privacy mechanism via its contraction properties. We first show that LDP constraints can be equivalently cast in terms of the contraction coefficient of the Eγ-divergence. We then use this equivalent formula to express LDP guarantees of privacy mechanisms in terms of contraction coefficients of arbitrary f -divergences. When combined with standard estimation-theoretic tools (such as Le Cam’s and Fano’s converse methods), this result allows us to study the trade-off between privacy and utility in several testing and minimax and Bayesian estimation problems.

[1]  Ilya Mironov,et al.  Rényi Differential Privacy , 2017, 2017 IEEE 30th Computer Security Foundations Symposium (CSF).

[2]  Gilles Barthe,et al.  Privacy Amplification by Mixing and Diffusion Mechanisms , 2019, NeurIPS.

[3]  Angelika Rohde,et al.  Geometrizing rates of convergence under local differential privacy constraints , 2020 .

[4]  Oliver Kosut,et al.  Three Variants of Differential Privacy: Lossless Conversion and Applications , 2021, IEEE Journal on Selected Areas in Information Theory.

[5]  Maxim Raginsky,et al.  Converses for distributed estimation via strong data processing inequalities , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[6]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[7]  Yihong Wu,et al.  Strong Data Processing Inequalities for Input Constrained Additive Noise Channels , 2015, IEEE Transactions on Information Theory.

[8]  Gaurav Kapoor,et al.  Protection Against Reconstruction and Its Applications in Private Federated Learning , 2018, ArXiv.

[9]  S L Warner,et al.  Randomized response: a survey technique for eliminating evasive answer bias. , 1965, Journal of the American Statistical Association.

[10]  Marco Gaboardi,et al.  Locally Private Mean Estimation: Z-test and Tight Confidence Intervals , 2018, AISTATS.

[11]  Ayfer Özgür,et al.  Fisher Information Under Local Differential Privacy , 2020, IEEE Journal on Selected Areas in Information Theory.

[12]  H. Vincent Poor,et al.  Channel Coding Rate in the Finite Blocklength Regime , 2010, IEEE Transactions on Information Theory.

[13]  P. Gács,et al.  Spreading of Sets in Product Spaces and Hypercontraction of the Markov Operator , 1976 .

[14]  Sofya Raskhodnikova,et al.  What Can We Learn Privately? , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[15]  Pramod Viswanath,et al.  Extremal Mechanisms for Local Differential Privacy , 2014, J. Mach. Learn. Res..

[16]  Moni Naor,et al.  Our Data, Ourselves: Privacy Via Distributed Noise Generation , 2006, EUROCRYPT.

[17]  Naresh Sharma,et al.  Fundamental bound on the reliability of quantum information transmission , 2012, Physical review letters.

[18]  Maxim Raginsky,et al.  Strong Data Processing Inequalities and $\Phi $ -Sobolev Inequalities for Discrete Channels , 2014, IEEE Transactions on Information Theory.

[19]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[20]  Yihong Wu,et al.  Dissipation of Information in Channels With Input Constraints , 2014, IEEE Transactions on Information Theory.

[21]  Alexandre V. Evfimievski,et al.  Limiting privacy breaches in privacy preserving data mining , 2003, PODS.

[22]  Himanshu Tyagi,et al.  Inference Under Information Constraints I: Lower Bounds From Chi-Square Contraction , 2018, IEEE Transactions on Information Theory.

[23]  Guy N. Rothblum,et al.  Concentrated Differential Privacy , 2016, ArXiv.

[24]  Joel E. Cohen,et al.  Relative entropy under mappings by stochastic matrices , 1993 .

[25]  Thomas Steinke,et al.  Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds , 2016, TCC.

[26]  John Duchi,et al.  Lower Bounds for Locally Private Estimation via Communication Complexity , 2019, COLT.

[27]  Peter Kairouz,et al.  Discrete Distribution Estimation under Local Privacy , 2016, ICML.

[28]  Yuhong Yang,et al.  Information-theoretic determination of minimax rates of convergence , 1999 .

[29]  S. M. Ali,et al.  A General Class of Coefficients of Divergence of One Distribution from Another , 1966 .

[30]  A. Makur,et al.  Comparison of Contraction Coefficients for f-Divergences , 2020, Problems of Information Transmission.

[31]  P. Moral,et al.  On contraction properties of Markov kernels , 2003 .

[32]  H. Witsenhausen.  ON SEQUENCES OF PAIRS OF DEPENDENT RANDOM VARIABLES , 1975 .

[33]  Yihong Wu,et al.  Strong data-processing inequalities for channels and Bayesian networks , 2015, 1508.06025.

[34]  Di Wang,et al.  On Sparse Linear Regression in the Local Differential Privacy Model , 2019, IEEE Transactions on Information Theory.

[35]  Gilles Barthe,et al.  Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences , 2018, NeurIPS.

[36]  Venkat Anantharam,et al.  On hypercontractivity and a data processing inequality , 2014, 2014 IEEE International Symposium on Information Theory.

[37]  Shahab Asoodeh,et al.  Privacy Analysis of Online Learning Algorithms via Contraction Coefficients , 2020, ArXiv.

[38]  Martin J. Wainwright,et al.  Local Privacy, Data Processing Inequalities, and Statistical Minimax Rates , 2013, 1302.3203.