Blind Over-the-Air Computation and Data Fusion via Provable Wirtinger Flow

Over-the-air computation (AirComp) shows great promise to support fast data fusion in Internet-of-Things (IoT) networks. AirComp typically computes desired functions of distributed sensing data by exploiting superposed data transmission in multiple access channels. To overcome its reliance on channel state information (CSI), this work proposes a novel blind over-the-air computation (BlairComp) without requiring CSI access, particularly for low complexity and low latency IoT networks. To solve the resulting non-convex optimization problem without the initialization dependency exhibited by the solutions of a number of recently proposed efficient algorithms, we develop a Wirtinger flow solution to the BlairComp problem based on random initialization. We establish the global convergence guarantee of Wirtinger flow with random initialization for BlairComp problem, which enjoys a model-agnostic and natural initialization implementation for practitioners with theoretical guarantees. Specifically, in the first stage of the algorithm, the iteration of randomly initialized Wirtinger flow given sufficient data samples can enter a local region that enjoys strong convexity and strong smoothness within a few iterations. We also prove the estimation error of BlairComp in the local region to be sufficiently small. We show that, at the second stage of the algorithm, its estimation error decays exponentially at a linear convergence rate.

[1]  John Wright,et al.  A Geometric Analysis of Phase Retrieval , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[2]  Deniz Gündüz,et al.  Collaborative Machine Learning at the Wireless Edge with Blind Transmitters , 2019, 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[3]  Michael Gastpar,et al.  Computation Over Multiple-Access Channels , 2007, IEEE Transactions on Information Theory.

[4]  Deniz Gündüz,et al.  Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[5]  Yuxin Chen,et al.  Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution , 2017, Found. Comput. Math..

[6]  Guo Wei,et al.  Over-the-air Computation for IoT Networks: Computing Multiple Functions with Antenna Arrays , 2018 .

[7]  H. Vincent Poor,et al.  Scheduling Policies for Federated Learning in Wireless Networks , 2019, IEEE Transactions on Communications.

[8]  Mohsen Guizani,et al.  Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications , 2015, IEEE Communications Surveys & Tutorials.

[9]  Kaibin Huang,et al.  Wirelessly Powered Data Aggregation for IoT via Over-the-Air Function Computation: Beamforming and Power Control , 2018, IEEE Transactions on Wireless Communications.

[10]  Kaibin Huang,et al.  Broadband Analog Aggregation for Low-Latency Federated Edge Learning , 2018, IEEE Transactions on Wireless Communications.

[11]  Jeffrey G. Andrews,et al.  What Will 5G Be? , 2014, IEEE Journal on Selected Areas in Communications.

[12]  Michael I. Jordan,et al.  How to Escape Saddle Points Efficiently , 2017, ICML.

[13]  Yuanming Shi,et al.  Nonconvex Demixing From Bilinear Measurements , 2018, IEEE Transactions on Signal Processing.

[14]  Slawomir Stanczak,et al.  Harnessing Interference for Analog Function Computation in Wireless Sensor Networks , 2013, IEEE Transactions on Signal Processing.

[15]  Thomas Strohmer,et al.  Regularized Gradient Descent: A Nonconvex Recipe for Fast Joint Blind Deconvolution and Demixing , 2017, ArXiv.

[16]  Michael Gastpar,et al.  Compute-and-Forward: Harnessing Interference Through Structured Codes , 2009, IEEE Transactions on Information Theory.

[17]  Kaibin Huang,et al.  MIMO Over-the-Air Computation for High-Mobility Multimodal Sensing , 2018, IEEE Internet of Things Journal.

[18]  Yuxin Chen,et al.  Gradient descent with random initialization: fast global convergence for nonconvex phase retrieval , 2018, Mathematical Programming.

[19]  Tao Jiang,et al.  Over-the-Air Computation via Intelligent Reflecting Surfaces , 2019, 2019 IEEE Global Communications Conference (GLOBECOM).

[20]  Nathan Srebro,et al.  Global Optimality of Local Search for Low Rank Matrix Recovery , 2016, NIPS.

[21]  Kaibin Huang,et al.  Reduced-Dimension Design of MIMO Over-the-Air Computing for Data Aggregation in Clustered IoT Networks , 2018, IEEE Transactions on Wireless Communications.

[22]  Kobi Cohen,et al.  A Sequential Gradient-Based Multiple Access for Distributed Learning over Fading Channels , 2019, 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[23]  Wei Chen,et al.  The Roadmap to 6G: AI Empowered Wireless Networks , 2019, IEEE Communications Magazine.

[24]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[25]  Furong Huang,et al.  Escaping From Saddle Points - Online Stochastic Gradient for Tensor Decomposition , 2015, COLT.

[26]  Slawomir Stanczak,et al.  On the Channel Estimation Effort for Analog Computation over Wireless Multiple-Access Channels , 2014, IEEE Wireless Communications Letters.

[27]  Yi Zheng,et al.  No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis , 2017, ICML.

[28]  Subhrakanti Dey,et al.  Distortion Outage Minimization and Diversity Order Analysis for Coherent Multiaccess , 2011, IEEE Transactions on Signal Processing.

[29]  Zhi Ding,et al.  Federated Learning via Over-the-Air Computation , 2018, IEEE Transactions on Wireless Communications.

[30]  A. Mukherjea,et al.  Real and Functional Analysis , 1978 .

[31]  Thomas Strohmer,et al.  Blind Deconvolution Meets Blind Demixing: Algorithms and Performance Bounds , 2015, IEEE Transactions on Information Theory.

[32]  Deniz Gündüz,et al.  One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis , 2020, ArXiv.

[33]  Yuanming Shi,et al.  Blind demixing for low-latency communication , 2018, 2018 IEEE Wireless Communications and Networking Conference (WCNC).

[34]  Andrea J. Goldsmith,et al.  Linear Coherent Decentralized Estimation , 2006, IEEE Transactions on Signal Processing.

[35]  Sriram Vishwanath,et al.  Communicating Linear Functions of Correlated Gaussian Sources Over a MAC , 2012, IEEE Transactions on Information Theory.

[36]  Slawomir Stanczak,et al.  Nomographic Functions: Efficient Computation in Clustered Gaussian Sensor Networks , 2013, IEEE Transactions on Wireless Communications.

[37]  Adel Javanmard,et al.  Theoretical Insights Into the Optimization Landscape of Over-Parameterized Shallow Neural Networks , 2017, IEEE Transactions on Information Theory.