Differentially Private Convex Optimization with Feasibility Guarantees

This paper develops a novel differentially private framework to solve convex optimization problems with sensitive optimization data and complex physical or operational constraints. Unlike standard noise-additive algorithms, that act primarily on the problem data, objective or solution, and disregard the problem constraints, this framework requires the optimization variables to be a function of the noise and exploits a chance-constrained problem reformulation with formal feasibility guarantees. The noise is calibrated to provide differential privacy for identity and linear queries on the optimization solution. For many applications, including resource allocation problems, the proposed framework provides a trade-off between the expected optimality loss and the variance of optimization results.

[1]  Jun Sakuma,et al.  Differentially Private Empirical Risk Minimization with Input Perturbation , 2017, DS.

[2]  Russell Bent,et al.  PowerModels.J1: An Open-Source Framework for Exploring Power Flow Formulations , 2017, 2018 Power Systems Computation Conference (PSCC).

[3]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[4]  Marco C. Campi,et al.  The Exact Feasibility of Randomized Solutions of Uncertain Convex Programs , 2008, SIAM J. Optim..

[5]  Ruiwei Jiang,et al.  Optimized Bonferroni approximations of distributionally robust joint chance constraints , 2019, Math. Program..

[6]  Ufuk Topcu,et al.  Differentially Private Distributed Constrained Optimization , 2014, IEEE Transactions on Automatic Control.

[7]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[8]  Quanyan Zhu,et al.  Dynamic Differential Privacy for ADMM-Based Distributed Classification Learning , 2017, IEEE Transactions on Information Forensics and Security.

[9]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[10]  Anand D. Sarwate,et al.  Stochastic gradient descent with differentially private updates , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[11]  Miao Pan,et al.  Differentially Private ADMM for Distributed Medical Machine Learning , 2019 .

[12]  Ufuk Topcu,et al.  Differentially private convex optimization with piecewise affine objectives , 2014, 53rd IEEE Conference on Decision and Control.

[13]  Vladimir Dvorkin,et al.  Differentially Private Distributed Optimal Power Flow , 2020, 2020 59th IEEE Conference on Decision and Control (CDC).

[14]  Catuscia Palamidessi,et al.  Broadening the Scope of Differential Privacy Using Metrics , 2013, Privacy Enhancing Technologies.

[15]  Steven H. Low,et al.  Differential Privacy of Aggregated DC Optimal Power Flow Data , 2019, 2019 American Control Conference (ACC).

[16]  Pascal Van Hentenryck,et al.  Differential Privacy of Hierarchical Census Data: An Optimization Approach , 2019, CP.

[17]  Alexander Shapiro,et al.  Convex Approximations of Chance Constrained Programs , 2006, SIAM J. Optim..

[18]  Ling Huang,et al.  Learning in a Large Function Space: Privacy-Preserving Mechanisms for SVM Learning , 2009, J. Priv. Confidentiality.

[19]  Jean-Philippe Vial,et al.  Robust Optimization , 2021, ICORES.

[20]  R. Tempo,et al.  On the sample complexity of randomized approaches to the analysis and design under uncertainty , 2010, Proceedings of the 2010 American Control Conference.

[21]  Andrés Muñoz.  Private linear programming without constraint violations , 2019 .

[22]  Raef Bassily,et al.  Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds , 2014, 1405.7085.

[23]  Ferdinando Fioretto,et al.  Bilevel Optimization for Differentially Private Optimization , 2020, ArXiv.

[24]  Daniel Kuhn,et al.  Generalized Gauss inequalities via semidefinite programming , 2015, Mathematical Programming.

[25]  Ferdinando Fioretto,et al.  Privacy-Preserving Power System Obfuscation: A Bilevel Optimization Approach , 2020, IEEE Transactions on Power Systems.

[26]  Andrew McGregor,et al.  Optimizing linear counting queries under differential privacy , 2009, PODS.

[27]  Anand D. Sarwate,et al.  Differentially Private Empirical Risk Minimization , 2009, J. Mach. Learn. Res..

[28]  Miao Pan,et al.  Optimal Differentially Private ADMM for Distributed Machine Learning , 2019, ArXiv.

[29]  Alexander J. Smola,et al.  Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlo , 2015, ICML.

[30]  Tim Roughgarden,et al.  Privately Solving Linear Programs , 2014, ICALP.

[31]  Kamalika Chaudhuri,et al.  Privacy-preserving logistic regression , 2008, NIPS.

[32]  John M. Abowd,et al.  The U.S. Census Bureau Adopts Differential Privacy , 2018, KDD.

[33]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[34]  Aaron Roth,et al.  Differentially private combinatorial optimization , 2009, SODA '10.

[35]  Wolfram Wiesemann,et al.  The decision rule approach to optimization under uncertainty: methodology and applications , 2018, Computational Management Science.

[36]  John Lygeros,et al.  On the Road Between Robust Optimization and the Scenario Approach for Chance Constrained Optimization Problems , 2014, IEEE Transactions on Automatic Control.

[37]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[38]  Kunal Talwar,et al.  Mechanism Design via Differential Privacy , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).