Information-Theoretic Foundations of Differential Privacy

We examine the information-theoretic foundations of the increasingly popular notion of differential privacy. We establish a connection between differential private mechanisms and the rate-distortion framework. Additionally, we also show how differentially private distributions arise out of the application of the Maximum Entropy Principle. This helps us locate differential privacy within the wider framework of information-theory and helps formalize some intuitive aspects of our understanding of differential privacy.

[1]  Poorvi L. Vora.  An Information-Theoretic Approach to Inference Attacks on Random Data Perturbation and a Related Privacy Measure , 2007, IEEE Transactions on Information Theory.

[2]  Ninghui Li,et al.  t-Closeness: Privacy Beyond k-Anonymity and l-Diversity , 2007, 2007 IEEE 23rd International Conference on Data Engineering.

[3]  Robin Milner,et al.  On Observing Nondeterminism and Concurrency , 1980, ICALP.

[4]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[5]  Michele Bezzi,et al.  An information theoretic approach for privacy metrics , 2010, Trans. Data Priv..

[6]  Darakhshan J. Mir.  Differentially-private learning and information theory , 2012, EDBT-ICDT '12.

[7]  Gilles Barthe,et al.  Information-Theoretic Bounds for Differentially Private Mechanisms , 2011, 2011 IEEE 24th Computer Security Foundations Symposium.

[8]  Cynthia Dwork,et al.  Differential Privacy , 2006, ICALP.

[9]  Mário S. Alvim,et al.  Differential Privacy: On the Trade-Off between Utility and Information Leakage , 2011, Formal Aspects in Security and Trust.

[10]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[11]  Kunal Talwar,et al.  Mechanism Design via Differential Privacy , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[12]  Toniann Pitassi,et al.  The Limits of Two-Party Differential Privacy , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[13]  H. Vincent Poor,et al.  A theory of utility and privacy of data sources , 2010, 2010 IEEE International Symposium on Information Theory.

[14]  Daniel A. Spielman,et al.  Spectral Graph Theory and its Applications , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[15]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[16]  Aaron D. Wyner,et al.  Coding Theorems for a Discrete Source With a Fidelity CriterionInstitute of Radio Engineers, International Convention Record, vol. 7, 1959. , 1993 .

[17]  Josep Domingo-Ferrer,et al.  From t-Closeness-Like Privacy to Postrandomization via Information Theory , 2010, IEEE Transactions on Knowledge and Data Engineering.

[18]  Yin Yang,et al.  Compressive mechanism: utilizing sparse representation in differential privacy , 2011, WPES.