Communication-Efficient Semihierarchical Federated Analytics in IoT Networks

The convergence of the Internet of Things (IoT) and data analytics has great potential to accelerate knowledge discovery, while the traditional approach of centralized data collection then processing is becoming infeasible in many applications due to efficiency and privacy concerns. Federated learning (FL) has emerged as a new paradigm that enables model learning across distributed IoT devices without sharing raw data. However, previous works on FL are either relying on a single central server or fully decentralized. In this article, we propose a semihierarchical federated analytics framework combining the advantages of the above architectures. The proposed framework leverages multiple edge servers for aggregating updates from IoT devices and fusing learned model weights without the need of cloud or a central server. Besides, we develop a new local client update rule to further improve the communication efficiency by reducing the communication rounds between IoT devices and edge servers. We analyze the convergence properties of the presenting approach and investigate its characteristics considering the effects of varying parameters, unreliable links, and packet loss. The experimental results demonstrate the effectiveness of our proposed methodology in providing communication-efficient, robust, and fault-tolerant data analytics to IoT networks.

[1]  Zhiguo Shi,et al.  Non-orthogonal Multiple Access assisted Federated Learning for UAV Swarms: An Approach of Latency Minimization , 2021, 2021 International Wireless Communications and Mobile Computing (IWCMC).

[2]  Swades De,et al.  Adaptive Multivariate Data Compression in Smart Metering Internet of Things , 2021, IEEE Transactions on Industrial Informatics.

[3]  Walid Saad,et al.  Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges , 2020, IEEE Communications Surveys & Tutorials.

[4]  Choong Seon Hong,et al.  Energy Efficient Federated Learning Over Wireless Communication Networks , 2019, IEEE Transactions on Wireless Communications.

[5]  Fenghua Zhu,et al.  Parallel Transportation Systems: Toward IoT-Enabled Smart Urban Traffic Control and Management , 2020, IEEE Transactions on Intelligent Transportation Systems.

[6]  Geyong Min,et al.  Communication-Efficient Federated Learning for Wireless Edge Intelligence in IoT , 2020, IEEE Internet of Things Journal.

[7]  Kerstin Thurow,et al.  A Flexible and Pervasive IoT-Based Healthcare Platform for Physiological and Environmental Parameters Monitoring , 2020, IEEE Internet of Things Journal.

[8]  Monica Nicoli,et al.  Federated Learning With Cooperating Devices: A Consensus Approach for Massive IoT Networks , 2019, IEEE Internet of Things Journal.

[9]  Li Chen,et al.  Accelerating Federated Learning via Momentum Gradient Descent , 2019, IEEE Transactions on Parallel and Distributed Systems.

[10]  Anit Kumar Sahu,et al.  Federated Learning: Challenges, Methods, and Future Directions , 2019, IEEE Signal Processing Magazine.

[11]  Xiaofei Wang,et al.  Convergence of Edge Computing and Deep Learning: A Comprehensive Survey , 2019, IEEE Communications Surveys & Tutorials.

[12]  Jun Zhang,et al.  Edge-Assisted Hierarchical Federated Learning with Non-IID Data , 2019, ArXiv.

[13]  Xin Zhou,et al.  Toward Computation Offloading in Edge Computing: A Survey , 2019, IEEE Access.

[14]  Hai Jin,et al.  Computation Offloading Toward Edge Computing , 2019, Proceedings of the IEEE.

[15]  Hubert Eichner,et al.  Towards Federated Learning at Scale: System Design , 2019, SysML.

[16]  Joseph Dureau,et al.  Federated Learning for Keyword Spotting , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[17]  Sebastian Caldas,et al.  Expanding the Reach of Federated Learning by Reducing Client Resource Requirements , 2018, ArXiv.

[18]  Martin Jaggi,et al.  COLA: Decentralized Linear Learning , 2018, NeurIPS.

[19]  Angelia Nedic,et al.  Distributed Optimization for Control , 2018, Annu. Rev. Control. Robotics Auton. Syst..

[20]  Giancarlo Fortino,et al.  A Novel Mobile and Hierarchical Data Transmission Architecture for Smart Factories , 2018, IEEE Transactions on Industrial Informatics.

[21]  Abhinav Vishnu,et al.  GossipGraD: Scalable Deep Learning using Gossip Communication based Asynchronous Gradient Descent , 2018, ArXiv.

[22]  William J. Dally,et al.  Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training , 2017, ICLR.

[23]  A. Salman Avestimehr,et al.  A Fundamental Tradeoff Between Computation and Communication in Distributed Computing , 2016, IEEE Transactions on Information Theory.

[24]  Kenneth Heafield,et al.  Sparse Communication for Distributed Gradient Descent , 2017, EMNLP.

[25]  Nei Kato,et al.  A Survey on Network Methodologies for Real-Time Analytics of Massive IoT Data and Open Research Issues , 2017, IEEE Communications Surveys & Tutorials.

[26]  Chun-Huat Heng,et al.  A Hybrid Data Compression Scheme for Power Reduction in Wireless Sensors for IoT , 2017, IEEE Transactions on Biomedical Circuits and Systems.

[27]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[28]  Matthieu Cord,et al.  Gossip training for deep learning , 2016, ArXiv.

[29]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[30]  Rakesh Kumar Jha,et al.  Device-to-Device Communication in Cellular Networks: A Survey , 2016, J. Netw. Comput. Appl..

[31]  Alexandre d'Aspremont,et al.  Regularized nonlinear acceleration , 2016, Mathematical Programming.

[32]  Angelia Nedic,et al.  Stochastic Gradient-Push for Strongly Convex Functions on Time-Varying Directed Graphs , 2014, IEEE Transactions on Automatic Control.

[33]  Euhanna Ghadimi,et al.  Global convergence of the Heavy-ball method for convex optimization , 2014, 2015 European Control Conference (ECC).

[34]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[35]  Yann LeCun,et al.  Deep learning with Elastic Averaging SGD , 2014, NIPS.

[36]  Shai Ben-David,et al.  Understanding Machine Learning: From Theory to Algorithms , 2014 .

[37]  José M. F. Moura,et al.  Fast Distributed Gradient Methods , 2011, IEEE Transactions on Automatic Control.

[38]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[39]  Toon van Waterschoot,et al.  Distributed estimation of static fields in wireless sensor networks using the finite element method , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  Per Christian Hansen,et al.  AIR Tools - A MATLAB package of algebraic iterative reconstruction methods , 2012, J. Comput. Appl. Math..

[41]  W. Marsden.  I and J , 2012 .

[42]  Annie I-An Chen,et al.  Fast Distributed First-Order Methods , 2012 .

[43]  Angelia Nedic,et al.  Asynchronous Broadcast-Based Convex Optimization Over a Network , 2011, IEEE Transactions on Automatic Control.

[44]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[45]  Philipp Birken,et al.  Numerical Linear Algebra , 2011, Encyclopedia of Parallel Computing.

[46]  Jeff Ahrenholz.  Comparison of CORE network emulation platforms , 2010, 2010 - MILCOM 2010 MILITARY COMMUNICATIONS CONFERENCE.

[47]  Panos M. Pardalos,et al.  Convex optimization theory , 2010, Optim. Methods Softw..

[48]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.

[49]  M. Glas,et al.  Principles of Computerized Tomographic Imaging , 2000 .