Lower Bound of Locally Differentially Private Sparse Covariance Matrix Estimation

In this paper, we study the sparse covariance matrix estimation problem in the local differential privacy model, and give a non-trivial lower bound on the non-interactive private minimax risk in the metric of squared spectral norm. We show that the lower bound is actually tight, as it matches a previous upper bound. Our main technique for achieving this lower bound is a general framework, called General Private Assouad Lemma, which is a considerable generalization of the previous private Assouad lemma and can be used as a general method for bounding the privateminimax risk ofmatrix-related estimation problems.

[1]  Úlfar Erlingsson,et al.  RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response , 2014, CCS.

[2]  P. Massart,et al.  Adaptive estimation of a quadratic functional by model selection , 2000 .

[3]  Vishesh Karwa,et al.  Finite Sample Differentially Private Confidence Intervals , 2017, ITCS.

[4]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[5]  Di Wang,et al.  On Sparse Linear Regression in the Local Differential Privacy Model , 2019, IEEE Transactions on Information Theory.

[6]  Feng Ruan,et al.  The Right Complexity Measure in Locally Private Estimation: It is not the Fisher Information , 2018, ArXiv.

[7]  Di Wang,et al.  Differentially Private High Dimensional Sparse Covariance Matrix Estimation , 2019, ArXiv.

[8]  Martin J. Wainwright,et al.  Local privacy and statistical minimax rates , 2013, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[9]  Janardhan Kulkarni,et al.  Locally Private Gaussian Estimation , 2018, NeurIPS.

[10]  Mengdi Huai,et al.  DIFFERENTIALLY PRIVATE SPARSE INVERSE COVARIANCE ESTIMATION , 2018, 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[11]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[12]  Harrison H. Zhou,et al.  Estimating Sparse Precision Matrix: Optimal Rates of Convergence and Adaptive Estimation , 2012, 1212.2882.

[13]  Harrison H. Zhou,et al.  OPTIMAL RATES OF CONVERGENCE FOR SPARSE COVARIANCE MATRIX ESTIMATION , 2012, 1302.3030.

[14]  Jonathan Ullman,et al.  Tight Lower Bounds for Locally Differentially Private Selection , 2018, ArXiv.

[15]  Joseph P. Near,et al.  Differential Privacy at Scale: Uber and Berkeley Collaboration , 2018 .

[16]  Yichen Wang,et al.  The Cost of Privacy: Optimal Rates of Convergence for Parameter Estimation with Differential Privacy , 2019, The Annals of Statistics.

[17]  Han Liu,et al.  Minimax-Optimal Privacy-Preserving Sparse PCA in Distributed Systems , 2018, AISTATS.

[18]  Martin J. Wainwright,et al.  Minimax Optimal Procedures for Locally Private Estimation , 2016, ArXiv.

[19]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[20]  John Duchi,et al.  Lower Bounds for Locally Private Estimation via Communication Complexity , 2019, COLT.

[21]  Jerry Li,et al.  Privately Learning High-Dimensional Distributions , 2018, COLT.

[22]  Li Zhang,et al.  Analyze gauss: optimal bounds for privacy-preserving principal component analysis , 2014, STOC.

[23]  Thomas Steinke,et al.  Tight Lower Bounds for Differentially Private Selection , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[24]  Jun Tang,et al.  Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10.12 , 2017, ArXiv.

[25]  Marco Gaboardi,et al.  Locally Private Mean Estimation: Z-test and Tight Confidence Intervals , 2018, AISTATS.