Accelerating Distributed Optimization via Over-the-Air Computing

Distributed optimization is ubiquitous in emerging applications, such as robust sensor network control, smart grid management, machine learning, resource slicing, and localization. However, the extensive data exchange among local and central nodes may cause a severe communication bottleneck. To overcome this challenge, over-the-air computing (AirComp) is a promising medium access technology, which exploits the superposition property of the wireless multiple access channel (MAC) and offers significant bandwidth savings. In this work, we propose an AirComp framework for general distributed convex optimization problems. Specifically, a distributed primal-dual (DPD) subgradient method is utilized for the optimization procedure. Under general assumptions, we prove that DPD-AirComp can asymptotically achieve zero expected constraint violation. Therefore, DPD-AirComp ensures the feasibility of the original problem, despite the presence of channel fading and additive noise. Moreover, with proper power control of the users’ signals, the expected non-zero optimality gap can also be mitigated. Two practical applications of the proposed framework are presented, namely, smart grid management and wireless resource allocation. Finally, numerical results confirm DPD-AirComp’s excellent performance, while it is also shown that DPD-AirComp converges an order of magnitude faster compared to two digital orthogonal multiple access schemes, specifically, time-division multiple access (TDMA), and orthogonal frequency-division multiple access (OFDMA).

[1]  Kaibin Huang,et al.  Distributed Over-the-Air Computing for Fast Distributed Optimization: Beamforming Design and Convergence Analysis , 2022, IEEE Journal on Selected Areas in Communications.

[2]  Bo Rong,et al.  6G: The Next Horizon: From Connected People and Things to Connected Intelligence , 2021, IEEE Wireless Communications.

[3]  Yuning Jiang,et al.  Over-the-Air Computation via Reconfigurable Intelligent Surface , 2021, IEEE Transactions on Communications.

[4]  Pavlos S. Bouzinis,et al.  Wireless Federated Learning (WFL) for 6G Networks—Part II: The Compute-Then-Transmit NOMA Paradigm , 2021, IEEE Communications Letters.

[5]  Panagiotis D. Diamantoulakis,et al.  Wireless Federated Learning (WFL) for 6G Networks⁴Part I: Research Challenges and Future Trends , 2021, IEEE Communications Letters.

[6]  Yue Wang,et al.  Joint Optimization of Communications and Federated Learning Over the Air , 2021, IEEE Transactions on Wireless Communications.

[7]  Walid Saad,et al.  Distributed Learning in Wireless Networks: Recent Progress and Future Challenges , 2021, IEEE Journal on Selected Areas in Communications.

[8]  Xuemin Shen,et al.  Integrating Over-the-Air Federated Learning and Non-Orthogonal Multiple Access: What Role Can RIS Play? , 2021, IEEE Transactions on Wireless Communications.

[9]  D. Niyato,et al.  Learning Rate Optimization for Federated Learning Exploiting Over-the-air Computation , 2021, IEEE Journal on Selected Areas in Communications.

[10]  Stefano Rini,et al.  Decentralized Optimization Over Noisy, Rate-Constrained Networks: Achieving Consensus by Communicating Differences , 2020, IEEE Journal on Selected Areas in Communications.

[11]  Yonina C. Eldar,et al.  Over-the-Air Federated Learning From Heterogeneous Data , 2020, IEEE Transactions on Signal Processing.

[12]  Deniz Gündüz,et al.  Federated Learning With Quantized Global Model Updates , 2020, ArXiv.

[13]  Deniz Gündüz,et al.  Decentralized SGD with Over-the-Air Computation , 2020, GLOBECOM 2020 - 2020 IEEE Global Communications Conference.

[14]  Meixia Tao,et al.  Gradient Statistics Aware Power Control for Over-the-Air Federated Learning , 2020, IEEE Transactions on Wireless Communications.

[15]  Osvaldo Simeone,et al.  Decentralized Federated Learning via SGD over Wireless D2D Networks , 2020, 2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[16]  Deniz Gündüz,et al.  One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis , 2020, IEEE Transactions on Wireless Communications.

[17]  Stefano Rini,et al.  Distributed Sub-gradient Algorithms with Limited Communications , 2019, 2019 53rd Asilomar Conference on Signals, Systems, and Computers.

[18]  Wanchun Liu,et al.  Over-the-Air Computation Systems: Optimization, Analysis and Scaling Laws , 2019, IEEE Transactions on Wireless Communications.

[19]  Deniz Gündüz,et al.  Federated Learning Over Wireless Fading Channels , 2019, IEEE Transactions on Wireless Communications.

[20]  Pingzhi Fan,et al.  6G Wireless Networks: Vision, Requirements, Architecture, and Key Technologies , 2019, IEEE Vehicular Technology Magazine.

[21]  B. Shihada,et al.  What should 6G be? , 2019, Nature Electronics.

[22]  Zhi Ding,et al.  Federated Learning via Over-the-Air Computation , 2018, IEEE Transactions on Wireless Communications.

[23]  Kaibin Huang,et al.  Broadband Analog Aggregation for Low-Latency Federated Edge Learning , 2018, IEEE Transactions on Wireless Communications.

[24]  Gabriela Hug,et al.  A Case for Nonconvex Distributed Optimization in Large-Scale Power Systems , 2017, IEEE Transactions on Power Systems.

[25]  Qingshan Liu,et al.  Distributed Optimization Based on a Multiagent System in the Presence of Communication Delays , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[26]  Xiangfeng Wang,et al.  Multi-Agent Distributed Optimization via Inexact Consensus ADMM , 2014, IEEE Transactions on Signal Processing.

[27]  Slawomir Stanczak,et al.  Harnessing Interference for Analog Function Computation in Wireless Sensor Networks , 2013, IEEE Transactions on Signal Processing.

[28]  Slawomir Stanczak,et al.  Robust Analog Function Computation via Wireless Multiple-Access Channels , 2012, IEEE Transactions on Communications.

[29]  Walid Saad,et al.  Economics of Electric Vehicle Charging: A Game Theoretic Approach , 2012, IEEE Transactions on Smart Grid.

[30]  Jeffrey H. Reed,et al.  Wireless distributed computing: a survey of research challenges , 2012, IEEE Communications Magazine.

[31]  Slawomir Stanczak,et al.  Analyzing the space of functions analog-computable via wireless multiple-access channels , 2011, 2011 8th International Symposium on Wireless Communication Systems.

[32]  Martin J. Wainwright,et al.  Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling , 2010, IEEE Transactions on Automatic Control.

[33]  Christos G. Cassandras,et al.  Asynchronous Distributed Optimization With Event-Driven Communication , 2010, IEEE Transactions on Automatic Control.

[34]  Angelia Nedic,et al.  Subgradient Methods for Saddle-Point Problems , 2009, J. Optimization Theory and Applications.

[35]  Asuman E. Ozdaglar,et al.  Approximate Primal Solutions and Rate Analysis for Dual Subgradient Methods , 2008, SIAM J. Optim..

[36]  Kurt M. Anstreicher,et al.  Linear Programming in O([n3/ln n]L) Operations , 1999, SIAM J. Optim..

[37]  Shuping Dang,et al.  Multiple Parallel Federated Learning via Over-the-Air Computation , 2022, IEEE Open Journal of the Communications Society.

[38]  Stephen P. Boyd,et al.  Subgradient Methods , 2007 .