Private Wireless Federated Learning with Anonymous Over-the-Air Computation

In conventional federated learning (FL), differential privacy (DP) guarantees can be obtained by injecting additional noise to local model updates before transmitting to the parameter server (PS). In the wireless FL scenario, we show that the privacy of the system can be boosted by exploiting over-the-air computation (OAC) and anonymizing the transmitting devices. In OAC, devices transmit their model updates simultaneously and in an uncoded fashion, resulting in a much more efficient use of the available spectrum. We further exploit OAC to provide anonymity for the transmitting devices. The proposed approach improves the performance of private wireless FL by reducing the amount of noise that must be injected.

[1]  Osvaldo Simeone,et al.  Privacy for Free: Wireless Federated Learning via Uncoded Transmission With Adaptive Power Control , 2020, IEEE Journal on Selected Areas in Communications.

[2]  Stefano Rini,et al.  Efficient Federated Learning over Multiple Access Channel with Differential Privacy Constraints , 2020, ArXiv.

[3]  Masahiro Morikura,et al.  Differentially Private AirComp Federated Learning with Power Adaptation Harnessing Receiver Noise , 2020, GLOBECOM 2020 - 2020 IEEE Global Communications Conference.

[4]  Michael Moeller,et al.  Inverting Gradients - How easy is it to break privacy in federated learning? , 2020, NeurIPS.

[5]  Ming Li,et al.  Wireless Federated Learning with Local Differential Privacy , 2020, 2020 IEEE International Symposium on Information Theory (ISIT).

[6]  Xinyuan Zhang,et al.  Local Differential Private Data Aggregation for Discrete Distribution Estimation , 2019, IEEE Transactions on Parallel and Distributed Systems.

[7]  Li Zhang,et al.  Rényi Differential Privacy of the Sampled Gaussian Mechanism , 2019, ArXiv.

[8]  Deniz Gündüz,et al.  Federated Learning Over Wireless Fading Channels , 2019, IEEE Transactions on Wireless Communications.

[9]  Deniz Gündüz,et al.  Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[10]  Song Han,et al.  Deep Leakage from Gradients , 2019, NeurIPS.

[11]  Ge Yu,et al.  Collecting and Analyzing Multidimensional Data with Local Differential Privacy , 2019, 2019 IEEE 35th International Conference on Data Engineering (ICDE).

[12]  Kaibin Huang,et al.  Broadband Analog Aggregation for Low-Latency Federated Edge Learning , 2018, IEEE Transactions on Wireless Communications.

[13]  Yu-Xiang Wang,et al.  Subsampled Rényi Differential Privacy and Analytical Moments Accountant , 2018, AISTATS.

[14]  Borja Balle,et al.  Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences , 2018, NeurIPS.

[15]  Vitaly Shmatikov,et al.  Exploiting Unintended Feature Leakage in Collaborative Learning , 2018, 2019 IEEE Symposium on Security and Privacy (SP).

[16]  Tassilo Klein,et al.  Differentially Private Federated Learning: A Client Level Perspective , 2017, ArXiv.

[17]  H. Brendan McMahan,et al.  Learning Differentially Private Recurrent Language Models , 2017, ICLR.

[18]  Ilya Mironov,et al.  Rényi Differential Privacy , 2017, 2017 IEEE 30th Computer Security Foundations Symposium (CSF).

[19]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[20]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[21]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[22]  Úlfar Erlingsson,et al.  RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response , 2014, CCS.