Differentially Private Submodular Maximization: Data Summarization in Disguise

Many data summarization applications are captured by the general framework of submodular maximization. As a consequence, a wide range of efficient approximation algorithms have been developed. However, when such applications involve sensitive data about individuals, their privacy concerns are not automatically addressed. To remedy this problem, we propose a general and systematic study of differentially private submodular maximization. We present privacypreserving algorithms for both monotone and non-monotone submodular maximization under cardinality, matroid, and p-extendible system constraints, with guarantees that are competitive with optimal solutions. Along the way, we analyze a new algorithm for non-monotone submodular maximization under a cardinality constraint, which is the first (even non-privately) to achieve a constant approximation ratio with a linear number of function evaluations. We additionally provide two concrete experiments to validate the efficacy of these algorithms.

[1]  Thomas Steinke,et al.  Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds , 2016, TCC.

[2]  R. Hardwarsing.  Stochastic Gradient Descent with Differentially Private Updates , 2018 .

[3]  Morteza Zadimoghaddam,et al.  Fast Distributed Submodular Cover: Public-Private Data Summarization , 2016, NIPS.

[4]  Kunal Talwar,et al.  Mechanism Design via Differential Privacy , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[5]  Thorsten Joachims,et al.  Temporal corpus summarization using submodular word coverage , 2012, CIKM '12.

[6]  Guy N. Rothblum,et al.  Boosting and Differential Privacy , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[7]  Julián Mestre,et al.  Greedy in Approximation Algorithms , 2006, ESA.

[8]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[9]  Andreas Krause,et al.  Near-optimal Nonmyopic Value of Information in Graphical Models , 2005, UAI.

[10]  Pravesh Kothari,et al.  Submodular functions are noise stable , 2012, SODA.

[11]  Kamalika Chaudhuri,et al.  The Large Margin Mechanism for Differentially Private Maximization , 2014, NIPS.

[12]  Andreas Krause,et al.  Budgeted Nonparametric Learning from Data Streams , 2010, ICML.

[13]  Jeff A. Bilmes,et al.  Submodularity for Data Selection in Statistical Machine Translation , 2014 .

[14]  Avinatan Hassidim,et al.  Submodular Optimization under Noise , 2016, COLT.

[15]  Andreas Krause,et al.  Lazier Than Lazy Greedy , 2014, AAAI.

[16]  Jan Vondrák,et al.  Maximizing a Monotone Submodular Function Subject to a Matroid Constraint , 2011, SIAM J. Comput..

[17]  Raef Bassily,et al.  Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds , 2014, 1405.7085.

[18]  Christos H. Papadimitriou,et al.  On the Hardness of Being Truthful , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[19]  Raef Bassily,et al.  Algorithmic stability for adaptive data analysis , 2015, STOC.

[20]  Éva Tardos,et al.  Maximizing the Spread of Influence through a Social Network , 2015, Theory Comput..

[21]  Joseph Naor,et al.  A Unified Continuous Greedy Algorithm for Submodular Maximization , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[22]  Joseph Naor,et al.  Submodular Maximization with Cardinality Constraints , 2014, SODA.

[23]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[24]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[25]  Kobbi Nissim,et al.  Differentially Private Release and Learning of Threshold Functions , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[26]  Andreas Krause,et al.  Near-Optimally Teaching the Crowd to Classify , 2014, ICML.

[27]  Vahab S. Mirrokni,et al.  Maximizing Non-Monotone Submodular Functions , 2011, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[28]  Hui Lin,et al.  A Class of Submodular Functions for Document Summarization , 2011, ACL.

[29]  Cynthia Dwork,et al.  Differential privacy and robust statistics , 2009, STOC '09.

[30]  Amos Beimel,et al.  Private Learning and Sanitization: Pure vs. Approximate Differential Privacy , 2013, APPROX-RANDOM.

[31]  Amin Karbasi,et al.  Greed Is Good: Near-Optimal Submodular Maximization via Greedy Optimization , 2017, COLT.

[32]  Baharan Mirzasoleiman,et al.  Fast Constrained Submodular Maximization: Personalized Data Summarization , 2016, ICML.

[33]  Aaron Roth,et al.  Differentially private combinatorial optimization , 2009, SODA '10.