Probabilistic Relational Reasoning for Differential Privacy

Differential privacy is a notion of confidentiality that allows useful computations on sensible data while protecting the privacy of individuals. Proving differential privacy is a difficult and error-prone task that calls for principled approaches and tool support. Approaches based on linear types and static analysis have recently emerged; however, an increasing number of programs achieve privacy using techniques that fall out of their scope. Examples include programs that aim for weaker, approximate differential privacy guarantees and programs that achieve differential privacy without using any standard mechanisms. Providing support for reasoning about the privacy of such programs has been an open problem. We report on CertiPriv, a machine-checked framework for reasoning about differential privacy built on top of the Coq proof assistant. The central component of CertiPriv is a quantitative extension of probabilistic relational Hoare logic that enables one to derive differential privacy guarantees for programs from first principles. We demonstrate the applicability of CertiPriv on a number of examples whose formal analysis is out of the reach of previous techniques. In particular, we provide the first machine-checked proofs of correctness of the Laplacian, Gaussian, and exponential mechanisms and of the privacy of randomized and streaming algorithms from the literature.

[1]  David Sands,et al.  Probabilistic noninterference for multi-threaded programs , 2000, Proceedings 13th IEEE Computer Security Foundations Workshop. CSFW-13.

[2]  Norman Ramsey,et al.  Stochastic lambda calculus and monads of probability distributions , 2002, POPL '02.

[3]  Mihir Bellare,et al.  The Security of Triple Encryption and a Framework for Code-Based Game-Playing Proofs , 2006, EUROCRYPT.

[4]  Gilles Barthe,et al.  Information-Theoretic Bounds for Differentially Private Mechanisms , 2011, 2011 IEEE 24th Computer Security Foundations Symposium.

[5]  Vitaly Shmatikov,et al.  Airavat: Security and Privacy for MapReduce , 2010, NSDI.

[6]  David Clark,et al.  A static analysis for quantifying information flow in a simple imperative language , 2007, J. Comput. Secur..

[7]  Kunal Talwar,et al.  Mechanism Design via Differential Privacy , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[8]  Wang Yi,et al.  Probabilistic Extensions of Process Algebras , 2001, Handbook of Process Algebra.

[9]  Frank McSherry,et al.  Privacy integrated queries: an extensible platform for privacy-preserving data analysis , 2009, SIGMOD Conference.

[10]  Christine Paulin-Mohring,et al.  Proofs of randomized algorithms in Coq , 2006, Sci. Comput. Program..

[11]  Benjamin C. Pierce,et al.  Distance makes the types grow stronger: a calculus for differential privacy , 2010, ICFP '10.

[12]  Eran Omri,et al.  Distributed Private Data Analysis: On Simultaneously Solving How and What , 2008, CRYPTO.

[13]  Nick Benton,et al.  Simple relational correctness proofs for static analyses and program transformations , 2004, POPL.

[14]  Dilsun Kirli Kaynar,et al.  Formal Verification of Differential Privacy for Interactive Systems , 2011, ArXiv.

[15]  Chris Hankin,et al.  Approximate non-interference , 2004 .

[16]  Benjamin Grégoire,et al.  Computer-Aided Security Proofs for the Working Cryptographer , 2011, CRYPTO.

[17]  Ashwin Machanavajjhala,et al.  No free lunch in data privacy , 2011, SIGMOD '11.

[18]  Roberto Segala,et al.  Approximated Computationally Bounded Simulation Relations for Probabilistic Automata , 2007, 20th IEEE Computer Security Foundations Symposium (CSF'07).

[19]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[20]  François Laviolette,et al.  Approximate Analysis of Probabilistic Processes: Logic, Simulation and Games , 2008, 2008 Fifth International Conference on Quantitative Evaluation of Systems.

[21]  Cynthia Dwork,et al.  Differential Privacy , 2006, ICALP.

[22]  Sumit Gulwani,et al.  Proving programs robust , 2011, ESEC/FSE '11.

[23]  Annabelle McIver,et al.  Probabilistic guarded commands mechanized in HOL , 2005, Theor. Comput. Sci..

[24]  Aaron Roth,et al.  Differentially private combinatorial optimization , 2009, SODA '10.

[25]  Elaine Shi,et al.  Private and Continual Release of Statistics , 2010, ICALP.

[26]  Omer Reingold,et al.  Computational Differential Privacy , 2009, CRYPTO.

[27]  Michael R. Clarkson,et al.  Hyperproperties , 2008, 2008 21st IEEE Computer Security Foundations Symposium.

[28]  Benjamin Grégoire,et al.  Formal certification of code-based cryptographic proofs , 2009, POPL '09.

[29]  Cynthia Dwork,et al.  Differential Privacy: A Survey of Results , 2008, TAMC.

[30]  Alexander Aiken,et al.  Secure Information Flow as a Safety Problem , 2005, SAS.