Differential Privacy Over Riemannian Manifolds

In this work we consider the problem of releasing a differentially private statistical summary that resides on a Riemannian manifold. We present an extension of the Laplace or K-norm mechanism that utilizes intrinsic distances and volumes on the manifold. We also consider in detail the specific case where the summary is the Fréchet mean of data residing on a manifold. We demonstrate that our mechanism is rate optimal and depends only on the dimension of the manifold, not on the dimension of any ambient space, while also showing how ignoring the manifold structure can decrease the utility of the sanitized summary. We illustrate our framework in two examples of particular interest in statistics: the space of symmetric positive definite matrices, which is used for covariance matrices, and the sphere, which can be used as a space for modeling discrete distributions.

[1]  Vishesh Karwa,et al.  Inference using noisy degrees: Differentially private $\beta$-model and synthetic graphs , 2012, 1205.4697.

[2]  Jonathan H. Manton,et al.  Riemannian Gaussian Distributions on the Space of Symmetric Positive Definite Matrices , 2015, IEEE Transactions on Information Theory.

[3]  Peter D. Hoff,et al.  Simulation of the Matrix Bingham–von Mises–Fisher Distribution, With Applications to Multivariate and Relational Data , 2007, 0712.4166.

[4]  Jonathan Ullman,et al.  Fingerprinting Codes and the Price of Approximate Differential Privacy , 2018, SIAM J. Comput..

[5]  Jonathan Ullman,et al.  CoinPress: Practical Private Mean and Covariance Estimation , 2020, NeurIPS.

[6]  B. Afsari.  Riemannian Lp center of mass: existence, uniqueness, and convexity , 2011 .

[7]  L. Wasserman,et al.  A Statistical Framework for Differential Privacy , 2008, 0811.2501.

[8]  Reinhard Wilhelm,et al.  Shape Analysis , 2000, CC.

[9]  Matthew Reimherr,et al.  Benefits and Pitfalls of the Exponential Mechanism with Applications to Hilbert Spaces and Functional PCA , 2019, ICML.

[10]  Di Wang,et al.  Principal Component Analysis in the Local Differential Privacy Model , 2019, IJCAI.

[11]  Kunal Talwar,et al.  On the geometry of differential privacy , 2009, STOC '10.

[12]  Marc Arnaudon,et al.  Riemannian Medians and Means With Applications to Radar Signal Processing , 2013, IEEE Journal of Selected Topics in Signal Processing.

[13]  I. Holopainen.  Riemannian Geometry , 1927, Nature.

[14]  Yannick Berthoumieu,et al.  Riemannian Laplace Distribution on the Space of Symmetric Positive Definite Matrices , 2016, Entropy.

[15]  Anuj Srivastava,et al.  Functional and Shape Data Analysis , 2016 .

[16]  K. Mardia,et al.  The von Mises–Fisher Matrix Distribution in Orientation Statistics , 1977 .

[17]  Cynthia Dwork,et al.  Practical privacy: the SuLQ framework , 2005, PODS.

[18]  H. Karcher.  Riemannian center of mass and mollifier smoothing , 1977 .

[19]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[20]  Sergei Vassilvitskii,et al.  Differentially Private Covariance Estimation , 2019, NeurIPS.

[21]  Krzysztof Choromanski,et al.  Differentially-private learning of low dimensional manifolds , 2016, Theor. Comput. Sci..

[22]  S. Helgason.  Differential Geometry, Lie Groups, and Symmetric Spaces , 1978 .

[23]  W. Kendall.  Probability, Convexity, and Harmonic Maps with Small Image I: Uniqueness and Fine Existence , 1990 .

[24]  Ramesh Raskar,et al.  Differentially Private Supervised Manifold Learning with Applications like Private Image Retrieval , 2021, ArXiv.

[25]  Themistocles M. Rassias,et al.  Introduction to Riemannian Manifolds , 2001 .

[26]  Matthew Reimherr,et al.  Formal Privacy for Functional Data with Gaussian Perturbations , 2017, ICML.

[27]  Jerry Li,et al.  Privately Learning High-Dimensional Distributions , 2018, COLT.

[28]  O. K. Yoon,et al.  Introduction to differentiable manifolds , 1993 .

[29]  Alexandre Barachant,et al.  Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review , 2017 .

[30]  Jiwu Huang,et al.  Fast and accurate Nearest Neighbor search in the manifolds of symmetric positive definite matrices , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[31]  Anand D. Sarwate,et al.  A near-optimal algorithm for differentially-private principal components , 2012, J. Mach. Learn. Res..

[32]  Leting Wu,et al.  Differential Privacy Preserving Spectral Graph Analysis , 2013, PAKDD.

[33]  Alfred O. Hero,et al.  Analysis of a privacy-preserving PCA algorithm using random matrix theory , 2016, 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP).