Privacy Preserving Collaborative Machine Learning

Collaborative machine learning is a promising paradigm that allows multiple participants to jointly train a machine learning model without exposing their private datasets to other parties. Although collaborative machine learning is more privacy-friendly compared with conventional machine learning methods, the intermediate model parameters exchanged among different participants in the training process may still reveal sensitive information about participants’ local datasets. In this paper, we introduce a novel privacypreserving collaborative machine learning mechanism by utilizing two non-colluding servers to perform secure aggregation of the intermediate parameters from participants. Compared with other existing solutions, our solution can achieve the same level of accuracy while incurring significantly lower computational cost. Received on 23 February 2021; accepted on 15 June 2021; published on 14 July 2021

[1]  Jorge Cortés,et al.  Differentially Private Distributed Convex Optimization via Functional Perturbation , 2015, IEEE Transactions on Control of Network Systems.

[2]  Payman Mohassel,et al.  SecureML: A System for Scalable Privacy-Preserving Machine Learning , 2017, 2017 IEEE Symposium on Security and Privacy (SP).

[3]  Vitaly Shmatikov,et al.  Privacy-preserving deep learning , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[4]  Magnus Egerstedt,et al.  Differentially private objective functions in distributed cloud-based optimization , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[5]  Gábor Danner,et al.  Fully Distributed Privacy Preserving Mini-batch Gradient Descent Learning , 2015, DAIS.

[6]  Stratis Ioannidis,et al.  Privacy-Preserving Ridge Regression on Hundreds of Millions of Records , 2013, 2013 IEEE Symposium on Security and Privacy.

[7]  Giuseppe Ateniese,et al.  Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning , 2017, CCS.

[8]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[9]  Yongqiang Wang,et al.  ADMM Based Privacy-Preserving Decentralized Optimization , 2017, IEEE Transactions on Information Forensics and Security.

[10]  Mariana Raykova,et al.  Secure Linear Regression on Vertically Partitioned Datasets , 2016, IACR Cryptol. ePrint Arch..

[11]  Jian Pei,et al.  Secure Skyline Queries on Cloud Platform , 2017, 2017 IEEE 33rd International Conference on Data Engineering (ICDE).

[12]  Amir Houmansadr,et al.  Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning , 2018, 2019 IEEE Symposium on Security and Privacy (SP).

[13]  Quanyan Zhu,et al.  Dynamic Differential Privacy for ADMM-Based Distributed Classification Learning , 2017, IEEE Transactions on Information Forensics and Security.

[14]  Michael Moeller,et al.  Inverting Gradients - How easy is it to break privacy in federated learning? , 2020, NeurIPS.

[15]  Kalyan Veeramachaneni,et al.  AnonML: Locally Private Machine Learning over a Network of Peers , 2017, 2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[16]  Mikhail Belkin,et al.  Crowd-ML: A Privacy-Preserving Learning Framework for a Crowd of Smart Devices , 2015, 2015 IEEE 35th International Conference on Distributed Computing Systems.

[17]  Sarvar Patel,et al.  Practical Secure Aggregation for Privacy-Preserving Machine Learning , 2017, IACR Cryptol. ePrint Arch..

[18]  Vitaly Shmatikov,et al.  Membership Inference Attacks Against Machine Learning Models , 2016, 2017 IEEE Symposium on Security and Privacy (SP).