Proving Differential Privacy in Hoare Logic

Differential privacy is a rigorous, worst-case notion of privacy-preserving computation. Informally, a probabilistic program is differentially private if the participation of a single individual in the input database has a limited effect on the program's distribution on outputs. More technically, differential privacy is a quantitative 2-safety property that bounds the distance between the output distributions of a probabilistic program on adjacent inputs. Like many 2-safety properties, differential privacy lies outside the scope of traditional verification techniques. Existing approaches to enforce privacy are based on intricate, non-conventional type systems, or customized relational logics. These approaches are difficult to implement and often cumbersome to use. We present an alternative approach that verifies differential privacy by standard, non-relational reasoning on non-probabilistic programs. Our approach transforms a probabilistic program into a non-probabilistic program which simulates two executions of the original program. We prove that if the target program is correct with respect to a Hoare specification, then the original probabilistic program is differentially private. We provide a variety of examples from the differential privacy literature to demonstrate the utility of our approach. Finally, we compare our approach with existing verification techniques for privacy.

[1]  Gilles Barthe,et al.  Probabilistic Relational Reasoning for Differential Privacy , 2012, TOPL.

[2]  Bernd Finkbeiner,et al.  Temporal Logics for Hyperproperties , 2013, POST.

[3]  Adam D. Smith,et al.  Differentially Private Feature Selection via Stability Arguments, and the Robustness of the Lasso , 2013, COLT.

[4]  Moni Naor,et al.  Differential privacy under continual observation , 2010, STOC '10.

[5]  Aaron Roth,et al.  Differentially private combinatorial optimization , 2009, SODA '10.

[6]  Pedro R. D'Argenio,et al.  Secure information flow by self-composition , 2004, Proceedings. 17th IEEE Computer Security Foundations Workshop, 2004..

[7]  Cynthia Dwork,et al.  Differential Privacy , 2006, ICALP.

[8]  Guy N. Rothblum,et al.  A Multiplicative Weights Mechanism for Privacy-Preserving Data Analysis , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[9]  Nick Benton,et al.  Simple relational correctness proofs for static analyses and program transformations , 2004, POPL.

[10]  Benjamin Grégoire,et al.  Computer-Aided Security Proofs for the Working Cryptographer , 2011, CRYPTO.

[11]  Reiner Hähnle,et al.  A Theorem Proving Approach to Analysis of Secure Information Flow , 2005, SPC.

[12]  Michael R. Clarkson,et al.  Belief semantics of authorization logic , 2013, CCS.

[13]  Cynthia Dwork,et al.  Practical privacy: the SuLQ framework , 2005, PODS.

[14]  Sanjeev Arora,et al.  The Multiplicative Weights Update Method: a Meta-Algorithm and Applications , 2012, Theory Comput..

[15]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[16]  Benjamin Grégoire,et al.  Formal certification of code-based cryptographic proofs , 2009, POPL '09.

[17]  Matteo Maffei,et al.  Differential Privacy by Typing in Security Protocols , 2013, 2013 IEEE 26th Computer Security Foundations Symposium.

[18]  Sumit Gulwani,et al.  Continuity analysis of programs , 2010, POPL '10.

[19]  Frank McSherry,et al.  Privacy integrated queries: an extensible platform for privacy-preserving data analysis , 2009, SIGMOD Conference.

[20]  Torben Amtoft,et al.  Information Flow Analysis in Logical Form , 2004, SAS.

[21]  Dave Clarke,et al.  Incremental Hyperproperty Model Checking via Games , 2013, NordSec.

[22]  Andreas Haeberlen,et al.  Linear dependent types for differential privacy , 2013, POPL.

[23]  2013 ACM SIGSAC Conference on Computer and Communications Security, CCS'13, Berlin, Germany, November 4-8, 2013 , 2013, Conference on Computer and Communications Security.

[24]  Katrina Ligett,et al.  A Simple and Practical Algorithm for Differentially Private Data Release , 2010, NIPS.

[25]  Vitaly Shmatikov,et al.  Airavat: Security and Privacy for MapReduce , 2010, NSDI.

[26]  Benjamin C. Pierce,et al.  Distance makes the types grow stronger: a calculus for differential privacy , 2010, ICFP '10.

[27]  Aaron Roth,et al.  Iterative Constructions and Private Data Release , 2011, TCC.

[28]  Cynthia Dwork,et al.  Differential Privacy: A Survey of Results , 2008, TAMC.

[29]  Amir Pnueli,et al.  CoVaC: Compiler Validation by Program Analysis of the Cross-Product , 2008, FM.

[30]  Elaine Shi,et al.  Private and Continual Release of Statistics , 2010, TSEC.

[31]  Dilsun Kirli Kaynar,et al.  Formal Verification of Differential Privacy for Interactive Systems , 2011, ArXiv.

[32]  Gilles Barthe,et al.  Beyond 2-Safety: Asymmetric Product Programs for Relational Program Verification , 2013, LFCS.

[33]  Gilles Barthe,et al.  Relational Verification Using Product Programs , 2011, FM.

[34]  Alexander Aiken,et al.  Secure Information Flow as a Safety Problem , 2005, SAS.

[35]  Kunal Talwar,et al.  Mechanism Design via Differential Privacy , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[36]  Cynthia Dwork,et al.  Differential privacy and robust statistics , 2009, STOC '09.

[37]  George Danezis,et al.  Verified Computational Differential Privacy with Applications to Smart Metering , 2013, 2013 IEEE 26th Computer Security Foundations Symposium.

[38]  Daniel A. Spielman,et al.  Spectral Graph Theory and its Applications , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[39]  Bernd Finkbeiner,et al.  Relational abstract interpretation for the verification of 2-hypersafety properties , 2013, CCS.