AHEAD: Adaptive Hierarchical Decomposition for Range Query under Local Differential Privacy

For protecting users' private data, local differential privacy (LDP) has been leveraged to provide the privacy-preserving range query, thus supporting further statistical analysis. However, existing LDP-based range query approaches are limited by their properties, ie, collecting user data according to a pre-defined structure. These static frameworks would incur excessive noise added to the aggregated data especially in the low privacy budget setting. In this work, we propose an Adaptive Hierarchical Decomposition (AHEAD) protocol, which adaptively and dynamically controls the built tree structure, so that the injected noise is well controlled for maintaining high utility. Furthermore, we derive a guideline for properly choosing parameters for AHEAD so that the overall utility can be consistently competitive while rigorously satisfying LDP. Leveraging multiple real and synthetic datasets, we extensively show the effectiveness of AHEAD in both low and high dimensional range query scenarios, as well as its advantages over the state-of-the-art methods. In addition, we provide a series of useful observations for deploying \myahead in practice.

[1]  Divesh Srivastava,et al.  Answering Range Queries Under Local Differential Privacy , 2018, Proc. VLDB Endow..

[2]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[3]  Yin Yang,et al.  Collecting and Analyzing Data from Smart Device Users with Local Differential Privacy , 2016, ArXiv.

[4]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[5]  Ming Li,et al.  PCKV: Locally Differentially Private Correlated Key-Value Data Collection with Optimized Utility , 2019, USENIX Security Symposium.

[6]  Ninghui Li,et al.  Answering Multi-Dimensional Range Queries under Local Differential Privacy , 2020, Proc. VLDB Endow..

[7]  Ninghui Li,et al.  Locally Differentially Private Protocols for Frequency Estimation , 2017, USENIX Security Symposium.

[8]  YangWei,et al.  Set-valued data publication with local privacy , 2020, VLDB 2020.

[9]  Mihaela van der Schaar,et al.  PATE-GAN: Generating Synthetic Data with Differential Privacy Guarantees , 2018, ICLR.

[10]  Martín Abadi,et al.  Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data , 2016, ICLR.

[11]  Yue Wang,et al.  A Data- and Workload-Aware Query Answering Algorithm for Range Queries Under Differential Privacy , 2014, Proc. VLDB Endow..

[12]  Xing Xie,et al.  PrivTree: A Differentially Private Algorithm for Hierarchical Decompositions , 2016, SIGMOD Conference.

[13]  Martin J. Wainwright,et al.  Local privacy and statistical minimax rates , 2013, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[14]  Ninghui Li,et al.  PriView: practical differentially private release of marginal contingency tables , 2014, SIGMOD Conference.

[15]  Yin Yang,et al.  Differentially private histogram publication , 2012, The VLDB Journal.

[16]  Haoran Li,et al.  DPCube: Differentially Private Histogram Release through Multidimensional Partitioning , 2012, Trans. Data Priv..

[17]  Úlfar Erlingsson,et al.  Building a RAPPOR with the Unknown: Privacy-Preserving Learning of Associations and Data Dictionaries , 2015, Proc. Priv. Enhancing Technol..

[18]  Torsten Suel,et al.  On Rectangular Partitionings in Two Dimensions: Algorithms, Complexity, and Applications , 1999, ICDT.

[19]  Divesh Srivastava,et al.  Marginal Release Under Local Differential Privacy , 2017, SIGMOD Conference.

[20]  Úlfar Erlingsson,et al.  RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response , 2014, CCS.

[21]  Dan Suciu,et al.  Boosting the accuracy of differentially private histograms through consistency , 2009, Proc. VLDB Endow..

[22]  Xinping Guan,et al.  Differential Private Noise Adding Mechanism and Its Application on Consensus Algorithm , 2016, IEEE Transactions on Signal Processing.

[23]  S L Warner,et al.  Randomized response: a survey technique for eliminating evasive answer bias. , 1965, Journal of the American Statistical Association.

[24]  Andrew McGregor,et al.  Optimizing linear counting queries under differential privacy , 2009, PODS.

[25]  Prateek Mittal,et al.  SecGraph: A Uniform and Open-source Evaluation System for Graph Data Anonymization and De-anonymization , 2015, USENIX Security Symposium.

[26]  Yannis E. Ioannidis,et al.  The History of Histograms (abridged) , 2003, VLDB.

[27]  Raef Bassily,et al.  Practical Locally Private Heavy Hitters , 2017, NIPS.

[28]  Jianliang Xu,et al.  Towards Accurate Histogram Publication under Differential Privacy , 2014, SDM.

[29]  Ninghui Li,et al.  Locally Differentially Private Frequency Estimation with Consistency , 2020, NDSS.

[30]  Liusheng Huang,et al.  Private Weighted Histogram Aggregation in Crowdsourcing , 2016, WASA.

[31]  Ninghui Li,et al.  Differentially private grids for geospatial data , 2012, 2013 IEEE 29th International Conference on Data Engineering (ICDE).

[32]  Yin Yang,et al.  Heavy Hitter Estimation over Set-Valued Data with Local Differential Privacy , 2016, CCS.

[33]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[34]  Ninghui Li,et al.  Differential Privacy: From Theory to Practice , 2016, Differential Privacy.

[35]  Yin Yang,et al.  Generating Synthetic Decentralized Social Graphs with Local Differential Privacy , 2017, CCS.

[36]  Ninghui Li,et al.  Locally Differentially Private Frequent Itemset Mining , 2018, 2018 IEEE Symposium on Security and Privacy (SP).

[37]  Claude Castelluccia,et al.  Differentially private sequential data publication via variable-length n-grams , 2012, CCS.

[38]  Stefan Axelsson,et al.  Paysim: a financial mobile money simulator for fraud detection , 2016 .

[39]  Prateek Mittal,et al.  On Your Social Network De-anonymizablity: Quantification and Large Scale Evaluation with Seed Knowledge , 2015, NDSS.

[40]  Ashwin Machanavajjhala,et al.  Principled Evaluation of Differentially Private Algorithms using DPBench , 2015, SIGMOD Conference.

[41]  Burton H. Bloom,et al.  Space/time trade-offs in hash coding with allowable errors , 1970, CACM.

[42]  Katrina Ligett,et al.  A Simple and Practical Algorithm for Differentially Private Data Release , 2010, NIPS.

[43]  Hongxia Jin,et al.  Private spatial data aggregation in the local setting , 2016, 2016 IEEE 32nd International Conference on Data Engineering (ICDE).

[44]  Jingren Zhou,et al.  Linear and Range Counting under Metric-based Local Differential Privacy , 2019, 2020 IEEE International Symposium on Information Theory (ISIT).

[45]  Yue Wang,et al.  A Data- and Workload-Aware Algorithm for Range Queries Under Differential Privacy , 2014, ArXiv.

[46]  Seth Neel,et al.  The Role of Interactivity in Local Differential Privacy , 2019, 2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS).

[47]  Divesh Srivastava,et al.  Differentially Private Spatial Decompositions , 2011, 2012 IEEE 28th International Conference on Data Engineering.

[48]  Lei Yang,et al.  Crowd-Empowered Privacy-Preserving Data Aggregation for Mobile Crowdsensing , 2018, MobiHoc.

[49]  Johannes Gehrke,et al.  Differential privacy via wavelet transforms , 2009, 2010 IEEE 26th International Conference on Data Engineering (ICDE 2010).

[50]  Xin Wang,et al.  Privacy-Preserving Distributed Machine Learning via Local Randomization and ADMM Perturbation , 2019, IEEE Transactions on Signal Processing.

[51]  Claude Castelluccia,et al.  Differentially Private Histogram Publishing through Lossy Compression , 2012, 2012 IEEE 12th International Conference on Data Mining.

[52]  Shaowei Wang,et al.  Set-valued Data Publication with Local Privacy: Tight Error Bounds and Efficient Mechanisms , 2020, Proc. VLDB Endow..

[53]  Janardhan Kulkarni,et al.  Collecting Telemetry Data Privately , 2017, NIPS.

[54]  Sofya Raskhodnikova,et al.  Smooth sensitivity and sampling in private data analysis , 2007, STOC '07.

[55]  Shouling Ji,et al.  Structural Data De-anonymization: Quantification, Practice, and Implications , 2014, CCS.

[56]  Aaron Roth,et al.  Local Differential Privacy for Evolving Data , 2018, NeurIPS.

[57]  Xi He,et al.  Kamino: Constraint-Aware Differentially Private Data Synthesis , 2020, Proc. VLDB Endow..

[58]  Haipei Sun,et al.  Analyzing Subgraph Statistics from Extended Local Views with Decentralized Differential Privacy , 2019, CCS.

[59]  Jiming Chen,et al.  CALM: Consistent Adaptive Local Marginal for Marginal Release under Local Differential Privacy , 2018, CCS.

[60]  Dawn Xiaodong Song,et al.  Practical Differential Privacy for SQL Queries Using Elastic Sensitivity , 2017, ArXiv.

[61]  Ninghui Li,et al.  Locally Differentially Private Heavy Hitter Identification , 2017, IEEE Transactions on Dependable and Secure Computing.

[62]  Ashwin Machanavajjhala,et al.  Optimizing error of high-dimensional statistical queries under differential privacy , 2018, Proc. VLDB Endow..

[63]  Aaron Roth,et al.  Exponential Separations in Local Differential Privacy , 2020, SODA.

[64]  Raef Bassily,et al.  Local, Private, Efficient Protocols for Succinct Histograms , 2015, STOC.

[65]  Yin Yang,et al.  PrivTrie: Effective Frequent Term Discovery under Local Differential Privacy , 2018, 2018 IEEE 34th International Conference on Data Engineering (ICDE).

[66]  Catuscia Palamidessi,et al.  Geo-indistinguishability: differential privacy for location-based systems , 2012, CCS.

[67]  Ninghui Li,et al.  Answering Multi-Dimensional Analytical Queries under Local Differential Privacy , 2019, SIGMOD Conference.

[68]  Sofya Raskhodnikova,et al.  What Can We Learn Privately? , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[69]  Cynthia Dwork,et al.  Differential Privacy , 2006, ICALP.

[70]  Aleksandar Nikolov,et al.  The power of factorization mechanisms in local and central differential privacy , 2019, STOC.

[71]  Ninghui Li,et al.  Understanding Hierarchical Methods for Differentially Private Histograms , 2013, Proc. VLDB Endow..

[72]  Philip S. Yu,et al.  $\textsf{LoPub}$ : High-Dimensional Crowdsourced Data Publication With Local Differential Privacy , 2016, IEEE Transactions on Information Forensics and Security.