Implementing Graph Neural Networks Over Wireless Networks via Over-the-Air Computing: A Joint Communication and Computation Framework

A Graph Neural Network (GNN) conducts the graph convolution for structured data and obtains the weighted sum over the vertices according to its graph structure. However, in the context of a wireless network, the traditional separate implementation of a GNN usually requires the full channel state information, which is hard to obtain in practice, especially for the underlying interference channels. On the other hand, Over-the-Air Computing (OAC) is an efficient analog wireless technique in which the weighted sum can be simultaneously calculated over an equivalent wireless superposition channel. Since the main goal of a distributed learning-based system is the fulfillment of the overall learning task instead of its communication rate, OAC is of great potential for implementing such a system. In this article, we exploit some specific features of the wireless interference graphs and propose a novel task-ori-ented OAC-based framework to deploy GNNs more efficiently in wireless networks. In particu-lar, we take advantage of the structural similarity between OAC and the graph convolution oper-ation over an interference graph, and the chan-nel prediction procedure can be merged into the weight updating procedure. Moreover, the inher-ent noise tolerance of a neural network further ensures its convergence and performance. We also conduct case studies based on the proposed framework and discuss the comprehensive future research directions and open problems.

[1]  Bradley Dirks,et al.  The minimal exponent and k-rationality for local complete intersections , 2022, Journal de l’École polytechnique — Mathématiques.

[2]  C. Zhong,et al.  Over-the-Air Split Machine Learning in Wireless MIMO Networks , 2022, IEEE Journal on Selected Areas in Communications.

[3]  Zhaohui Yang,et al.  Over-the-Air Split Learning with MIMO-Based Neural Network and Constellation-Based Activation , 2022, 2022 IEEE 32nd International Workshop on Machine Learning for Signal Processing (MLSP).

[4]  H. Dai,et al.  Privacy-Preserving Decentralized Inference with Graph Neural Networks in Wireless Networks , 2022, IEEE Transactions on Wireless Communications.

[5]  Derrick Wing Kwan Ng,et al.  Edge Learning for B5G Networks With Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing , 2022, IEEE Journal of Selected Topics in Signal Processing.

[6]  Colin N. Jones,et al.  Over-the-Air Federated Learning via Second-Order Optimization , 2022, IEEE Transactions on Wireless Communications.

[7]  Jia Guo,et al.  Learning Power Allocation for Multi-Cell-Multi-User Systems With Heterogeneous Graph Neural Networks , 2022, IEEE Transactions on Wireless Communications.

[8]  Shiwen He,et al.  An Overview on the Application of Graph Neural Networks in Wireless Networks , 2021, IEEE Open Journal of the Communications Society.

[9]  K. Mikolajczyk,et al.  AirNet: Neural Network Transmission over the Air , 2021, 2022 IEEE International Symposium on Information Theory (ISIT).

[10]  Yizhou Sun,et al.  Heterogeneous Network Representation Learning: A Unified Framework With Survey and Benchmark , 2020, IEEE Transactions on Knowledge and Data Engineering.

[11]  Alejandro Ribeiro,et al.  Optimal Wireless Resource Allocation With Random Edge Graph Neural Networks , 2019, IEEE Transactions on Signal Processing.

[12]  Stephan ten Brink,et al.  Deep Learning Based Communication Over the Air , 2017, IEEE Journal of Selected Topics in Signal Processing.

[13]  Wei Zhang,et al.  Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent , 2017, NIPS.

[14]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[15]  Dong Yu,et al.  1-bit stochastic gradient descent and its application to data-parallel distributed training of speech DNNs , 2014, INTERSPEECH.

[16]  Changyang She,et al.  Distributed Graph Neural Networks for Optimizing Wireless Networks: Message Passing Over-the-Air , 2022 .