Model Checking Differentially Private Properties

We introduce the branching time temporal logic Open image in new window for specifying differential privacy. Several differentially private mechanisms are formalized as Markov chains or Markov decision processes. Using our formal models, subtle privacy conditions are specified by Open image in new window . In order to verify privacy properties automatically, model checking problems are investigated. We give a model checking algorithm for Markov chains. Model checking Open image in new window properties on Markov decision processes however is shown to be undecidable.

[1]  Gilles Barthe,et al.  Probabilistic Relational Reasoning for Differential Privacy , 2012, TOPL.

[2]  Tim Roughgarden,et al.  Universally utility-maximizing privacy mechanisms , 2008, STOC '09.

[3]  Azaria Paz,et al.  Introduction to probabilistic automata (Computer science and applied mathematics) , 1971 .

[4]  Andreas Haeberlen,et al.  A framework for adaptive differential privacy , 2017, Proc. ACM Program. Lang..

[5]  Cynthia Dwork,et al.  Differential Privacy , 2006, ICALP.

[6]  Federico Olmedo,et al.  Probabilistic Reasoning for Differential Privacy , 2012 .

[7]  Andreas Haeberlen,et al.  Linear dependent types for differential privacy , 2013, POPL.

[8]  Grégoire Sutre,et al.  An Optimal Automata Approach to LTL Model Checking of Probabilistic Systems , 2003, LPAR.

[9]  Andrea Bianco,et al.  Model Checking of Probabalistic and Nondeterministic Systems , 1995, FSTTCS.

[10]  Jun Tang,et al.  Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10.12 , 2017, ArXiv.

[11]  Stephan Merz,et al.  Model Checking , 2000 .

[12]  Gilles Barthe,et al.  Differentially Private Bayesian Programming , 2016, CCS.

[13]  Azaria Paz,et al.  Probabilistic automata , 2003 .

[14]  Dilsun Kirli Kaynar,et al.  Formal Verification of Differential Privacy for Interactive Systems , 2011, ArXiv.

[15]  Zohar Manna,et al.  The Temporal Logic of Reactive and Concurrent Systems , 1991, Springer New York.

[16]  Bengt Jonsson,et al.  A logic for reasoning about time and reliability , 1990, Formal Aspects of Computing.

[17]  Mihalis Yannakakis,et al.  The complexity of probabilistic verification , 1995, JACM.

[18]  Dale Miller,et al.  Preserving differential privacy under finite-precision semantics , 2013, Theor. Comput. Sci..

[19]  Ilya Mironov,et al.  On significance of the least significant bits for differential privacy , 2012, CCS.

[20]  George Danezis,et al.  Verified Computational Differential Privacy with Applications to Smart Metering , 2013, 2013 IEEE 26th Computer Security Foundations Symposium.

[21]  Benjamin C. Pierce,et al.  Distance makes the types grow stronger: a calculus for differential privacy , 2010, ICFP '10.

[22]  Philip S. Yu,et al.  Privacy-preserving data publishing: A survey of recent developments , 2010, CSUR.

[23]  Christel Baier,et al.  Principles of model checking , 2008 .

[24]  Mário S. Alvim,et al.  On the information leakage of differentially-private mechanisms , 2015, J. Comput. Secur..

[25]  Wen-Guey Tzeng,et al.  A Polynomial-Time Algorithm for the Equivalence of Probabilistic Automata , 1992, SIAM J. Comput..

[26]  Christel Baier,et al.  Markov Chains and Unambiguous Büchi Automata , 2016, CAV.

[27]  Gilles Barthe,et al.  Proving Differential Privacy in Hoare Logic , 2014, 2014 IEEE 27th Computer Security Foundations Symposium.

[28]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[29]  Christel Baier,et al.  Probabilistic ω-automata , 2012, JACM.

[30]  Pierre-Yves Strub,et al.  Advanced Probabilistic Couplings for Differential Privacy , 2016, CCS.

[31]  Gilles Barthe,et al.  Higher-Order Approximate Relational Refinement Types for Mechanism Design and Differential Privacy , 2014, POPL.

[32]  Charles Elkan,et al.  Differential Privacy and Machine Learning: a Survey and Review , 2014, ArXiv.

[33]  Benjamin Grégoire,et al.  Proving Differential Privacy via Probabilistic Couplings , 2016, 2016 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[34]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[35]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[36]  Danfeng Zhang,et al.  LightDP: towards automating differential privacy proofs , 2016, POPL.