Broadband Digital Over-the-Air Computation for Asynchronous Federated Edge Learning

This paper presents the first broadband digital overthe-air computation (AirComp) system for phase asynchronous OFDM-based federated edge learning systems. Existing analog AirComp systems often assume perfect phase alignment via channel precoding and utilize uncoded analog modulation for model aggregation. In contrast, our digital AirComp system leverages digital modulation and channel codes to overcome phase asynchrony, thereby achieving accurate model aggregation in the asynchronous multi-user OFDM systems. To realize a digital AirComp system, we propose a non-orthogonal multiple access protocol that allows simultaneous transmissions from multiple edge devices, and present a joint channel decoding and aggregation (Jt-CDA) decoder (i.e., full-state joint decoder). To reduce the computation complexity, we further present a reducedcomplexity Jt-CDA decoder (i.e., reduced-state joint decoder), and its arithmetic sum bit error rate performance is similar to that of the full-state joint decoder for most signal-to-noise ratio (SNR) regimes. Simulation results on test accuracy (of CIFAR10 dataset) versus SNR show that: 1) analog AirComp systems are sensitive to phase asynchrony under practical setup, and the test accuracy performance exhibits an error floor even at high SNR regime; 2) our digital AirComp system outperforms an analog AirComp system by at least 1.5 times when SNR≥9dB, demonstrating the advantage of digital AirComp in asynchronous multi-user OFDM systems.

[1]  Zhibin Wang,et al.  Federated Learning via Intelligent Reflecting Surface , 2020, IEEE Transactions on Wireless Communications.

[2]  Soung Chang Liew,et al.  Reliable Physical-Layer Network Coding Supporting Real Applications , 2017, IEEE Transactions on Mobile Computing.

[3]  Deniz Gündüz,et al.  Over-the-Air Machine Learning at the Wireless Edge , 2019, 2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[4]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[5]  Slawomir Stanczak,et al.  Robust Analog Function Computation via Wireless Multiple-Access Channels , 2012, IEEE Transactions on Communications.

[6]  Deniz Gündüz,et al.  Denoising Noisy Neural Networks: A Bayesian Approach with Compensation , 2021, ArXiv.

[7]  Yonina C. Eldar,et al.  Over-the-Air Federated Learning From Heterogeneous Data , 2020, IEEE Transactions on Signal Processing.

[8]  Deniz Gündüz,et al.  Bayesian Over-The-Air Computation , 2021, ArXiv.

[9]  Michael Gastpar,et al.  Source-Channel Communication in Sensor Networks , 2003, IPSN.

[10]  Omid Salehi-Abari,et al.  AirShare: Distributed coherent transmission made seamless , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[11]  Zhi Ding,et al.  Federated Learning via Over-the-Air Computation , 2018, IEEE Transactions on Wireless Communications.

[12]  Slawomir Stanczak,et al.  Analog computation over the wireless channel: A proof of concept , 2014, IEEE SENSORS 2014 Proceedings.

[13]  Xiangyu Zhang,et al.  ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design , 2018, ECCV.

[14]  Omid Salehi-Abari,et al.  Over-the-air Function Computation in Sensor Networks , 2016, ArXiv.

[15]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[16]  Kaibin Huang,et al.  Broadband Analog Aggregation for Low-Latency Federated Edge Learning , 2018, IEEE Transactions on Wireless Communications.

[17]  Michael Gastpar,et al.  Computation Over Multiple-Access Channels , 2007, IEEE Transactions on Information Theory.

[18]  Giuseppe Caire,et al.  AirSync: Enabling Distributed Multiuser MIMO With Full Spatial Multiplexing , 2012, IEEE/ACM Transactions on Networking.

[19]  Soung Chang Liew,et al.  Hot topic: physical-layer network coding , 2006, MobiCom '06.

[20]  Ezzeldin Hamed,et al.  Chorus: truly distributed distributed-MIMO , 2018, SIGCOMM.

[21]  Deniz Gündüz,et al.  Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[22]  Jinho Choi,et al.  Convolutional Codes in Two-Way Relay Networks with Physical-Layer Network Coding , 2010, IEEE Transactions on Wireless Communications.

[23]  Deniz Gündüz,et al.  Federated Edge Learning with Misaligned Over-The-Air Computation , 2021, 2021 IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[24]  Deniz Gündüz,et al.  One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis , 2020, IEEE Transactions on Wireless Communications.

[25]  Osvaldo Simeone,et al.  Decentralized Federated Learning via SGD over Wireless D2D Networks , 2020, 2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).