Jointly Private Convex Programming

In this paper we present an extremely general method for approximately solving a large family of convex programs where the solution can be divided between different agents, subject to joint differential privacy. This class includes multi-commodity flow problems, general allocation problems, and multi-dimensional knapsack problems, among other examples. The accuracy of our algorithm depends on the \emph{number} of constraints that bind between individuals, but crucially, is \emph{nearly independent} of the number of primal variables and hence the number of agents who make up the problem. As the number of agents in a problem grows, the error we introduce often becomes negligible. We also consider the setting where agents are strategic and have preferences over their part of the solution. For any convex program in this class that maximizes \emph{social welfare}, there is a generic reduction that makes the corresponding optimization \emph{approximately dominant strategy truthful} by charging agents prices for resources as a function of the approximately optimal dual variables, which are themselves computed under differential privacy. Our results substantially expand the class of problems that are known to be solvable under both privacy and incentive constraints.

[1]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[2]  Aaron Roth,et al.  Asymptotically truthful equilibrium selection in large congestion games , 2013, EC.

[3]  Faruk Gul,et al.  WALRASIAN EQUILIBRIUM WITH GROSS SUBSTITUTES , 1999 .

[4]  Anand D. Sarwate,et al.  Differentially Private Empirical Risk Minimization , 2009, J. Mach. Learn. Res..

[5]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[6]  Stephen Chong,et al.  Truthful mechanisms for agents that value privacy , 2011, EC.

[7]  Moni Naor,et al.  On the complexity of differentially private data release: efficient algorithms and hardness results , 2009, STOC '09.

[8]  Aaron Roth,et al.  Differentially private combinatorial optimization , 2009, SODA '10.

[9]  Sampath Kannan,et al.  The Exponential Mechanism for Social Welfare: Private, Truthful, and Nearly Optimal , 2012, 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science.

[10]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[11]  Prateek Jain,et al.  (Near) Dimension Independent Risk Bounds for Differentially Private Learning , 2014, ICML.

[12]  Daniel Kifer,et al.  Private Convex Optimization for Empirical Risk Minimization with Applications to High-dimensional Regression , 2012, COLT.

[13]  Justin Hsu,et al.  Differential privacy for the analyst via private equilibrium computation , 2012, STOC '13.

[14]  Pravesh Kothari,et al.  25th Annual Conference on Learning Theory Differentially Private Online Learning , 2022 .

[15]  Aaron Roth,et al.  Privacy and Truthful Equilibrium Selection for Aggregative Games , 2014, WINE.

[16]  Aaron Roth,et al.  Mechanism design in large games: incentives and privacy , 2012, ITCS.

[17]  Moni Naor,et al.  The Privacy of the Analyst and the Power of the State , 2012, FOCS.

[18]  Alessandro Panconesi,et al.  Concentration of Measure for the Analysis of Randomized Algorithms , 2009 .

[19]  Guy N. Rothblum,et al.  Boosting and Differential Privacy , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[20]  Tim Roughgarden,et al.  Private matchings and allocations , 2013, SIAM J. Comput..

[21]  Raef Bassily,et al.  Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds , 2014, 1405.7085.

[22]  Yoav Freund,et al.  Game theory, on-line prediction and boosting , 1996, COLT '96.

[23]  Moshe Tennenholtz,et al.  Approximately optimal mechanism design via differential privacy , 2010, ITCS '12.

[24]  Sofya Raskhodnikova,et al.  Smooth sensitivity and sampling in private data analysis , 2007, STOC '07.

[25]  David Xiao,et al.  Is privacy compatible with truthfulness? , 2013, ITCS '13.

[26]  M. Slater Lagrange Multipliers Revisited , 2014 .

[27]  Irit Dinur,et al.  Revealing information while preserving privacy , 2003, PODS.

[28]  Cynthia Dwork,et al.  Practical privacy: the SuLQ framework , 2005, PODS.

[29]  Daniel Kifer,et al.  Private Convex Empirical Risk Minimization and High-dimensional Regression , 2012, COLT 2012.

[30]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[31]  Kunal Talwar,et al.  Mechanism Design via Differential Privacy , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[32]  Tim Roughgarden,et al.  Privately Solving Linear Programs , 2014, ICALP.

[33]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.