An Introduction to Communication Efficient Edge Machine Learning

In the near future, Internet-of-Things (IoT) is expected to connect billions of devices (e.g., smartphones and sensors), which generate massive real-time data at the network edge. Intelligence can be distilled from the data to support next-generation AI-powered applications, which is called edge machine learning. One challenge faced by edge learning is the communication bottleneck, which is caused by the transmission of high-dimensional data from many edge devices to edge servers for learning. Traditional wireless techniques focusing only on efficient radio access are ineffective in tackling the challenge. Solutions should be based on a new approach that seamlessly integrates communication and computation. This has led to the emergence of a new cross-disciplinary paradigm called communication efficient edge learning. The main theme in the area is to design new communication techniques and protocols for efficient implementation of different distributed learning frameworks (i.e., federated learning) in wireless networks. This article provides an overview of the emerging area by introducing new design principles, discussing promising research opportunities, and providing design examples based on recent work.

[1]  Robert W. Heath,et al.  An Overview of Signal Processing Techniques for Millimeter Wave MIMO Systems , 2015, IEEE Journal of Selected Topics in Signal Processing.

[2]  Kaibin Huang,et al.  Towards an Intelligent Edge: Wireless Communication Meets Machine Learning , 2018, ArXiv.

[3]  Gustavo de Veciana,et al.  Joint Scheduling of URLLC and eMBB Traffic in 5G Wireless Networks , 2017, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[4]  Kaibin Huang,et al.  Wirelessly Powered Data Aggregation for IoT via Over-the-Air Function Computation: Beamforming and Power Control , 2018, IEEE Transactions on Wireless Communications.

[5]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[6]  Yi Zhang,et al.  Incorporating Diversity and Density in Active Learning for Relevance Feedback , 2007, ECIR.

[7]  Michael Gastpar,et al.  Compute-and-Forward: Harnessing Interference Through Structured Codes , 2009, IEEE Transactions on Information Theory.

[8]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[9]  Jeffrey G. Andrews,et al.  Downlink SDMA with Limited Feedback in Interference-Limited Wireless Networks , 2011, IEEE Transactions on Wireless Communications.

[10]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[11]  Uri Erez,et al.  Achieving 1/2 log (1+SNR) on the AWGN channel with lattice encoding and decoding , 2004, IEEE Transactions on Information Theory.

[12]  Robert W. Heath,et al.  Foundations of MIMO Communication , 2018 .

[13]  W. Marsden I and J , 2012 .

[14]  K. B. Letaief,et al.  A Survey on Mobile Edge Computing: The Communication Perspective , 2017, IEEE Communications Surveys & Tutorials.

[15]  Abbas Jamalipour,et al.  Wireless communications , 2005, GLOBECOM '05. IEEE Global Telecommunications Conference, 2005..

[16]  Kenneth Heafield,et al.  Sparse Communication for Distributed Gradient Descent , 2017, EMNLP.

[17]  Jun Zhang,et al.  Data-Importance Aware User Scheduling for Communication-Efficient Edge Machine Learning , 2019, IEEE Transactions on Cognitive Communications and Networking.

[18]  Brian L. Evans,et al.  Optimal Downlink OFDMA Resource Allocation with Linear Complexity to Maximize Ergodic Rates , 2008, IEEE Transactions on Wireless Communications.

[19]  Sumei Sun,et al.  Power Efficient Resource Allocation for Downlink OFDMA Relay Cellular Networks , 2012, IEEE Transactions on Signal Processing.

[20]  Kin K. Leung,et al.  Energy-Efficient Radio Resource Allocation for Federated Edge Learning , 2019, 2020 IEEE International Conference on Communications Workshops (ICC Workshops).

[21]  Kaibin Huang,et al.  Reduced-Dimension Design of MIMO Over-the-Air Computing for Data Aggregation in Clustered IoT Networks , 2018, IEEE Transactions on Wireless Communications.

[22]  William J. Dally,et al.  Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training , 2017, ICLR.

[23]  Sanjay Shakkottai,et al.  Joint Scheduling of URLLC and eMBB Traffic in 5G Wireless Networks , 2020, IEEE/ACM Transactions on Networking.

[24]  Kaibin Huang,et al.  High-Dimensional Stochastic Gradient Quantization for Communication-Efficient Edge Learning , 2019, 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[25]  Zhi Ding,et al.  Federated Learning Based on Over-the-Air Computation , 2019, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[26]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[27]  Kaibin Huang,et al.  Automatic Recognition of Space-Time Constellations by Learning on the Grassmann Manifold , 2018, IEEE Transactions on Signal Processing.

[28]  Mohamed-Slim Alouini,et al.  Performance analysis of multiuser selection diversity , 2004, 2004 IEEE International Conference on Communications (IEEE Cat. No.04CH37577).

[29]  Irene Y. H. Gu,et al.  Online subspace learning on Grassmann manifold for moving object tracking in video , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[30]  Kin K. Leung,et al.  When Edge Meets Learning: Adaptive Control for Resource-Constrained Distributed Machine Learning , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[31]  Deniz Gündüz,et al.  Deep Joint Source-Channel Coding for Wireless Image Transmission , 2019, IEEE Transactions on Cognitive Communications and Networking.

[32]  Sergio Verdú,et al.  Randomly spread CDMA: asymptotics via statistical physics , 2005, IEEE Transactions on Information Theory.

[33]  Mohamed-Slim Alouini,et al.  Area spectral efficiency of cellular mobile radio systems , 1999 .

[34]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[35]  Zhi-Hua Zhou,et al.  Active Query Driven by Uncertainty and Diversity for Incremental Multi-label Learning , 2013, 2013 IEEE 13th International Conference on Data Mining.

[36]  David D. Lewis,et al.  Heterogeneous Uncertainty Sampling for Supervised Learning , 1994, ICML.

[37]  Kaibin Huang,et al.  MIMO Over-the-Air Computation for High-Mobility Multimodal Sensing , 2018, IEEE Internet of Things Journal.

[38]  Robert W. Heath,et al.  Constructing Packings in Grassmannian Manifolds via Alternating Projection , 2007, Exp. Math..

[39]  Kaibin Huang,et al.  Broadband Analog Aggregation for Low-Latency Federated Edge Learning , 2018, IEEE Transactions on Wireless Communications.