More Than Privacy: Applying Differential Privacy in Key Areas of Artificial Intelligence

Artificial Intelligence (AI) has attracted a great deal of attention in recent years. However, alongside all its advancements, problems have also emerged, such as privacy violations, security issues and model fairness. Differential privacy, as a promising mathematical model, has several attractive properties that can help solve these problems, making it quite a valuable tool. For this reason, differential privacy has been broadly applied in AI but to date, no study has documented which differential privacy mechanisms can or have been leveraged to overcome its issues or the properties that make this possible. In this paper, we show that differential privacy can do more than just privacy preservation. It can also be used to improve security, stabilize learning, build fair models, and impose composition in selected areas of AI. With a focus on regular machine learning, distributed machine learning, deep learning, and multi-agent systems, the purpose of this article is to deliver a new view on many possibilities for improving AI performance with differential privacy techniques.

[1]  Maoguo Gong,et al.  Preserving differential privacy in deep neural networks with relevance-based adaptive noise imposition , 2020, Neural Networks.

[2]  Qiang Yang,et al.  Active Transfer Learning for Cross-System Recommendation , 2013, AAAI.

[3]  Sofya Raskhodnikova,et al.  What Can We Learn Privately? , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[4]  Athanasios V. Vasilakos,et al.  A Survey of Self-Organization Mechanisms in Multiagent Systems , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[5]  John Langford,et al.  A Reductions Approach to Fair Classification , 2018, ICML.

[6]  Ju Ren,et al.  GANobfuscator: Mitigating Information Leakage Under GAN via Differential Privacy , 2019, IEEE Transactions on Information Forensics and Security.

[7]  Marc Sebban,et al.  Differentially Private Optimal Transport: Application to Domain Adaptation , 2019, IJCAI.

[8]  Toniann Pitassi,et al.  Preserving Statistical Validity in Adaptive Data Analysis , 2014, STOC.

[9]  Gulshan Kumar,et al.  A Survey of Deep Learning and Its Applications: A New Paradigm to Machine Learning , 2019, Archives of Computational Methods in Engineering.

[10]  Somesh Jha,et al.  Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting , 2017, 2018 IEEE 31st Computer Security Foundations Symposium (CSF).

[11]  Rui Zhang,et al.  A Hybrid Approach to Privacy-Preserving Federated Learning , 2018, Informatik Spektrum.

[12]  Pablo Hernandez-Leal,et al.  Uncertainty-Aware Action Advising for Deep Reinforcement Learning Agents , 2020, AAAI.

[13]  Daniel Sheldon,et al.  Differentially Private Bayesian Inference for Exponential Families , 2018, NeurIPS.

[14]  Tao Qin,et al.  Learning to Teach , 2018, ICLR.

[15]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[16]  Antti Honkela,et al.  Differentially private Bayesian learning on distributed data , 2017, NIPS.

[17]  Jiayu Zhou,et al.  Privacy-Preserving Distributed Multi-Task Learning with Asynchronous Updates , 2017, KDD.

[18]  Zhenkai Liang,et al.  Neural Network Inversion in Adversarial Setting via Background Knowledge Alignment , 2019, CCS.

[19]  Yang Wang,et al.  Differentially Private Hypothesis Transfer Learning , 2018, ECML/PKDD.

[20]  D. Fitch,et al.  Review of "Algorithms of oppression: how search engines reinforce racism," by Noble, S. U. (2018). New York, New York: NYU Press. , 2018, CDQR.

[21]  Felipe Leno da Silva,et al.  A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems , 2019, J. Artif. Intell. Res..

[22]  Gilles Barthe,et al.  Probabilistic Relational Reasoning for Differential Privacy , 2012, TOPL.

[23]  L. S. Shapley,et al.  College Admissions and the Stability of Marriage , 2013, Am. Math. Mon..

[24]  Lingxiao Wang,et al.  Distributed Learning without Distress: Privacy-Preserving Empirical Risk Minimization , 2018, NeurIPS.

[25]  Sanjiv Kumar,et al.  cpSGD: Communication-efficient and differentially-private distributed SGD , 2018, NeurIPS.

[26]  Stefano Ermon,et al.  Generative Adversarial Imitation Learning , 2016, NIPS.

[27]  Mark Klein,et al.  Auctions and bidding: A guide for computer scientists , 2011, CSUR.

[28]  Aaron Roth,et al.  Asymptotically truthful equilibrium selection in large congestion games , 2013, EC.

[29]  R. Rosenthal,et al.  More on the "anti-folk theorem" , 1989 .

[30]  Aaron Roth,et al.  An Antifolk Theorem for Large Repeated Games , 2016, ACM Trans. Economics and Comput..

[31]  Katrina Ligett,et al.  A Simple and Practical Algorithm for Differentially Private Data Release , 2010, NIPS.

[32]  Tian Li,et al.  Fair Resource Allocation in Federated Learning , 2019, ICLR.

[33]  Matthew E. Taylor,et al.  A survey and critique of multiagent deep reinforcement learning , 2019, Autonomous Agents and Multi-Agent Systems.

[34]  Saul Perlmutter,et al.  Blind analysis: Hide results to seek the truth , 2015, Nature.

[35]  Miao Pan,et al.  Differentially Private and Fair Classification via Calibrated Functional Mechanism , 2020, AAAI.

[36]  Aaron Roth,et al.  Privacy and mechanism design , 2013, SECO.

[37]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Qiang Yang,et al.  Privacy-Preserving Stacking with Application to Cross-organizational Diabetes Prediction , 2019, IJCAI.

[39]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[40]  Martín Abadi,et al.  Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data , 2016, ICLR.

[41]  Nicholas R. Jennings,et al.  Intelligent agents: theory and practice , 1995, The Knowledge Engineering Review.

[42]  Toniann Pitassi,et al.  The reusable holdout: Preserving validity in adaptive data analysis , 2015, Science.

[43]  Dan Qu,et al.  Towards end-to-end speech recognition with transfer learning , 2018, EURASIP Journal on Audio, Speech, and Music Processing.

[44]  Yanjiao Chen,et al.  Privacy-Preserving Collaborative Deep Learning With Unreliable Participants , 2020, IEEE Transactions on Information Forensics and Security.

[45]  Bogdan Gabrys,et al.  Metalearning: a survey of trends and technologies , 2013, Artificial Intelligence Review.

[46]  Shiho Moriai,et al.  Privacy-Preserving Deep Learning via Additively Homomorphic Encryption , 2018, IEEE Transactions on Information Forensics and Security.

[47]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[48]  Bin Li,et al.  Selling Multiple Items via Social Networks , 2018, AAMAS.

[49]  Cynthia Dwork,et al.  Differential Privacy , 2006, ICALP.

[50]  Aaron Roth.  Differential privacy, equilibrium, and efficient allocation of resources , 2013, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[51]  David Evans,et al.  Evaluating Differentially Private Machine Learning in Practice , 2019, USENIX Security Symposium.

[52]  Wenqi Wei,et al.  Private and Truthful Aggregative Game for Large-Scale Spectrum Sharing , 2017, IEEE Journal on Selected Areas in Communications.

[53]  S. Noble.  Algorithms of Oppression: How Search Engines Reinforce Racism , 2018 .

[54]  Payman Mohassel,et al.  SecureML: A System for Scalable Privacy-Preserving Machine Learning , 2017, 2017 IEEE Symposium on Security and Privacy (SP).

[55]  Jie Cui,et al.  Differentially Private Double Spectrum Auction With Approximate Social Welfare Maximization , 2019, IEEE Transactions on Information Forensics and Security.

[56]  Nan Duan,et al.  Progress in Neural NLP: Modeling, Learning, and Reasoning , 2020, Engineering.

[57]  David Eckhoff,et al.  Metrics : a Systematic Survey , 2018 .

[58]  Raef Bassily,et al.  Private Stochastic Convex Optimization with Optimal Rates , 2019, NeurIPS.

[59]  Felipe Leno da Silva,et al.  Simultaneously Learning and Advising in Multiagent Reinforcement Learning , 2017, AAMAS.

[60]  Nidhi Hegde,et al.  Privacy-Preserving Q-Learning with Functional Noise in Continuous Spaces , 2019, NeurIPS.

[61]  Amir Houmansadr,et al.  Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning , 2018, 2019 IEEE Symposium on Security and Privacy (SP).

[62]  Jeffrey Li,et al.  Differentially Private Meta-Learning , 2020, ICLR.

[63]  Somesh Jha,et al.  Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures , 2015, CCS.

[64]  Di Wang,et al.  Differentially Private Empirical Risk Minimization Revisited: Faster and More General , 2018, NIPS.

[65]  Aaron Roth,et al.  Mechanism design in large games: incentives and privacy , 2012, ITCS.

[66]  Tassilo Klein,et al.  Differentially Private Federated Learning: A Client Level Perspective , 2017, ArXiv.

[67]  Pavlos Moraitis,et al.  Argumentation-based Negotiation with Incomplete Opponent Profiles , 2019, AAMAS.

[68]  Ricardo Vilalta,et al.  A Perspective View and Survey of Meta-Learning , 2002, Artificial Intelligence Review.

[69]  Yang Liu,et al.  Heterogeneous Gaussian Mechanism: Preserving Differential Privacy in Deep Learning with Provable Robustness , 2019, IJCAI.

[70]  Gerhard Weiss,et al.  Multiagent Learning: Basics, Challenges, and Prospects , 2012, AI Mag..

[71]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[72]  J. Roemer,et al.  Equality of Opportunity , 2013 .

[73]  Guihai Chen,et al.  Differentially private spectrum auction with approximate revenue maximization , 2014, MobiHoc '14.

[74]  Blaise Agüera y Arcas,et al.  Federated Learning of Deep Networks using Model Averaging , 2016, ArXiv.

[75]  Vasant Honavar,et al.  A Conceptual Framework for Secrecy-preserving Reasoning in Knowledge Bases , 2014, ACM Trans. Comput. Log..

[76]  Rui Zhang,et al.  A Hybrid Approach to Privacy-Preserving Federated Learning , 2019, AISec@CCS.

[77]  Feng Yan,et al.  LEASGD: an Efficient and Privacy-Preserving Decentralized Algorithm for Distributed Learning , 2018, ArXiv.

[78]  Úlfar Erlingsson,et al.  Scalable Private Learning with PATE , 2018, ICLR.

[79]  Matthew E. Taylor,et al.  Autonomously Reusing Knowledge in Multiagent Reinforcement Learning , 2018, IJCAI.

[80]  Qiang He,et al.  An Agent-Based Integrated Self-Evolving Service Composition Approach in Networked Environments , 2019, IEEE Transactions on Services Computing.

[81]  Robert H. Deng,et al.  Privacy-Preserving Reinforcement Learning Design for Patient-Centric Dynamic Treatment Regimes , 2019, IEEE Transactions on Emerging Topics in Computing.

[82]  Namil Kim,et al.  Pixel-Level Domain Transfer , 2016, ECCV.

[83]  Qing-Long Han,et al.  A survey on recent advances in distributed sampled-data cooperative control of multi-agent systems , 2018, Neurocomputing.

[84]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[85]  Jonathan Ullman,et al.  Preventing False Discovery in Interactive Data Analysis Is Hard , 2014, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[86]  Anand D. Sarwate,et al.  Differentially Private Empirical Risk Minimization , 2009, J. Mach. Learn. Res..

[87]  Kristina Lerman,et al.  A Survey on Bias and Fairness in Machine Learning , 2019, ACM Comput. Surv..

[88]  Kang G. Shin,et al.  Differentially private and strategy-proof spectrum auction with approximate revenue maximization , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[89]  Cynthia Dwork,et al.  Fairness Under Composition , 2018, ITCS.

[90]  Andreas Haeberlen,et al.  Fuzzi: a three-level logic for differential privacy , 2019, Proc. ACM Program. Lang..

[91]  Toniann Pitassi,et al.  Fairness through awareness , 2011, ITCS '12.

[92]  Chuang Gan,et al.  The Neuro-Symbolic Concept Learner: Interpreting Scenes Words and Sentences from Natural Supervision , 2019, ICLR.

[93]  Bin Li,et al.  Mechanism Design in Social Networks , 2016, AAAI.

[94]  Sofya Raskhodnikova,et al.  Testing the Lipschitz Property over Product Distributions with Applications to Data Privacy , 2013, TCC.

[95]  Toniann Pitassi,et al.  Learning Fair Representations , 2013, ICML.

[96]  Matt J. Kusner,et al.  Counterfactual Fairness , 2017, NIPS.

[97]  Mihaela van der Schaar,et al.  PATE-GAN: Generating Synthetic Data with Differential Privacy Guarantees , 2018, ICLR.

[98]  Vitaly Feldman,et al.  PAC learning with stable and private predictions , 2019, COLT 2020.

[99]  G. Owen,et al.  Game Theory (2nd Ed.). , 1983 .

[100]  Shigenobu Kobayashi,et al.  Privacy-preserving reinforcement learning , 2008, ICML '08.

[101]  Minjie Zhang,et al.  A Hybrid Multiagent Framework With Q-Learning for Power Grid Systems Restoration , 2011, IEEE Transactions on Power Systems.

[102]  Dayong Ye,et al.  A Self-Adaptive Sleep/Wake-Up Scheduling Approach for Wireless Sensor Networks , 2018, IEEE Transactions on Cybernetics.

[103]  Daniel Sheldon,et al.  Differentially Private Bayesian Linear Regression , 2019, NeurIPS.

[104]  Philip S. Yu,et al.  Fairness in Semi-Supervised Learning: Unlabeled Data Help to Reduce Discrimination , 2020, IEEE Transactions on Knowledge and Data Engineering.

[105]  Frank McSherry,et al.  Privacy integrated queries: an extensible platform for privacy-preserving data analysis , 2009, SIGMOD Conference.

[106]  Yuzhe Tang,et al.  PADS: Privacy-Preserving Auction Design for Allocating Dynamically Priced Cloud Resources , 2017, 2017 IEEE 3rd International Conference on Collaboration and Internet Computing (CIC).

[107]  Andrew A. Tawfik,et al.  Using a Recommendation System to Support Problem Solving and Case-Based Reasoning Retrieval , 2018, Technol. Knowl. Learn..

[108]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[109]  Christos Dimitrakakis,et al.  Algorithms for Differentially Private Multi-Armed Bandits , 2015, AAAI.

[110]  Mehryar Mohri,et al.  Agnostic Federated Learning , 2019, ICML.

[111]  Justin Schwartz.  Engineering , 1929, Nature.

[112]  Justin Hsu,et al.  Differential privacy for the analyst via private equilibrium computation , 2012, STOC '13.

[113]  Christos Dimitrakakis,et al.  Achieving Privacy in the Adversarial Multi-Armed Bandit , 2017, AAAI.

[114]  Elaine Shi,et al.  Private and Continual Release of Statistics , 2010, TSEC.

[115]  Miroslav Dudík,et al.  Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need? , 2018, CHI.

[116]  Jonathan P. How,et al.  Learning to Teach in Cooperative Multiagent Reinforcement Learning , 2018, AAAI.

[117]  Prateek Mittal,et al.  Analyzing Federated Learning through an Adversarial Lens , 2018, ICML.

[118]  Somesh Jha,et al.  Privacy in Pharmacogenetics: An End-to-End Case Study of Personalized Warfarin Dosing , 2014, USENIX Security Symposium.

[119]  Jun Zhao.  Distributed Deep Learning under Differential Privacy with the Teacher-Student Paradigm , 2018, AAAI Workshops.

[120]  Tianjian Chen,et al.  Federated Machine Learning: Concept and Applications , 2019 .

[121]  Sergey Levine,et al.  Online Meta-Learning , 2019, ICML.

[122]  Peter Henderson,et al.  An Introduction to Deep Reinforcement Learning , 2018, Found. Trends Mach. Learn..

[123]  Nicolas Maudet,et al.  Efficiency, Sequenceability and Deal-Optimality in Fair Division of Indivisible Goods , 2018, AAMAS.

[124]  Haixu Tang,et al.  Learning your identity and disease from research papers: information leaks in genome wide association study , 2009, CCS.

[125]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[126]  Nathan Srebro,et al.  Equality of Opportunity in Supervised Learning , 2016, NIPS.

[127]  Philip S. Yu,et al.  Applying Differential Privacy Mechanism in Artificial Intelligence , 2019, 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS).

[128]  Varun Gupta,et al.  On the Compatibility of Privacy and Fairness , 2019, UMAP.

[129]  Giuseppe Ateniese,et al.  Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning , 2017, CCS.

[130]  Vitaly Feldman,et al.  Privacy-preserving Prediction , 2018, COLT.

[131]  Ufuk Topcu,et al.  An approximately truthful mechanism for electric vehicle charging via joint differential privacy , 2015, 2015 American Control Conference (ACC).

[132]  Feng Wu,et al.  Privacy-Preserving Policy Iteration for Decentralized POMDPs , 2018, AAAI.

[133]  Philip S. Yu,et al.  Differentially Private Data Publishing and Analysis: A Survey , 2017, IEEE Transactions on Knowledge and Data Engineering.

[134]  Pascal Van Hentenryck,et al.  Privacy-Preserving Federated Data Sharing , 2019, AAMAS.

[135]  Alexandra Chouldechova,et al.  The Frontiers of Fairness in Machine Learning , 2018, ArXiv.

[136]  Sheng Zhong,et al.  Joint Differentially Private Gale–Shapley Mechanisms for Location Privacy Protection in Mobile Traffic Offloading Systems , 2016, IEEE Journal on Selected Areas in Communications.

[137]  Kunal Talwar,et al.  Mechanism Design via Differential Privacy , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[138]  Xintao Wu,et al.  Achieving Differential Privacy and Fairness in Logistic Regression , 2019, WWW.

[139]  Joshua Achiam,et al.  On First-Order Meta-Learning Algorithms , 2018, ArXiv.

[140]  Philip S. Yu,et al.  Differentially Private Malicious Agent Avoidance in Multiagent Advising Learning , 2020, IEEE Transactions on Cybernetics.

[141]  Reuben Binns,et al.  Fairness in Machine Learning: Lessons from Political Philosophy , 2017, FAT.

[142]  Aaron Roth,et al.  Differentially Private Fair Learning , 2018, ICML.

[143]  Shaojie Tang,et al.  Designing differentially private spectrum auction mechanisms , 2016, Wirel. Networks.

[144]  Vitaly Shmatikov,et al.  Membership Inference Attacks Against Machine Learning Models , 2016, 2017 IEEE Symposium on Security and Privacy (SP).

[145]  Michael Wooldridge,et al.  The dMARS Architecture: A Specification of the Distributed Multi-Agent Reasoning System , 2004, Autonomous Agents and Multi-Agent Systems.

[146]  Mohammad Al-Rubaie,et al.  Privacy-Preserving Machine Learning: Threats and Solutions , 2018, IEEE Security & Privacy.

[147]  Vitaly Shmatikov,et al.  Privacy-preserving deep learning , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[148]  Úlfar Erlingsson,et al.  RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response , 2014, CCS.

[149]  Éva Tardos,et al.  Learning and Efficiency in Games with Dynamic Population , 2015, SODA.

[150]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[151]  André Elisseeff,et al.  Stability and Generalization , 2002, J. Mach. Learn. Res..

[152]  Diego Perez Liebana,et al.  Teaching on a Budget in Multi-Agent Deep Reinforcement Learning , 2019, 2019 IEEE Conference on Games (CoG).

[153]  James R. Foulds,et al.  On the Theory and Practice of Privacy-Preserving Bayesian Data Analysis , 2016, UAI.

[154]  Ameet Talwalkar,et al.  Federated Multi-Task Learning , 2017, NIPS.

[155]  Curtis R. Taylor,et al.  The Economics of Privacy , 2016 .