Aggregation Strategy on Federated Machine Learning Algorithm for Collaborative Predictive Maintenance

Industry 4.0 lets the industry build compact, precise, and connected assets and also has made modern industrial assets a massive source of data that can be used in process optimization, defining product quality, and predictive maintenance (PM). Large amounts of data are collected from machines, processed, and analyzed by different machine learning (ML) algorithms to achieve effective PM. These machines, assumed as edge devices, transmit their data readings to the cloud for processing and modeling. Transmitting massive amounts of data between edge and cloud is costly, increases latency, and causes privacy concerns. To address this issue, efforts have been made to use edge computing in PM applications., reducing data transmission costs and increasing processing speed. Federated learning (FL) has been proposed a mechanism that provides the ability to create a model from distributed data in edge, fog, and cloud layers without violating privacy and offers new opportunities for a collaborative approach to PM applications. However, FL has challenges in confronting with asset management in the industry, especially in the PM applications, which need to be considered in order to be fully compatible with these applications. This study describes distributed ML for PM applications and proposes two federated algorithms: Federated support vector machine (FedSVM) with memory for anomaly detection and federated long-short term memory (FedLSTM) for remaining useful life (RUL) estimation that enables factories at the fog level to maximize their PM models’ accuracy without compromising their privacy. A global model at the cloud level has also been generated based on these algorithms. We have evaluated the approach using the Commercial Modular Aero-Propulsion System Simulation (CMAPSS) dataset to predict engines’ RUL Experimental results demonstrate the advantage of FedSVM and FedLSTM in terms of model accuracy, model convergence time, and network usage resources.

[1]  Hongbo Zhu,et al.  Optimized Edge Aggregation for Hierarchical Federated Learning , 2021, 2021 IEEE 94th Vehicular Technology Conference (VTC2021-Fall).

[2]  Farshad Firouzi,et al.  The convergence and interplay of edge, fog, and cloud in the AI-driven Internet of Things (IoT) , 2021, Inf. Syst..

[3]  Hangyu Zhu,et al.  Federated Learning on Non-IID Data: A Survey , 2021, Neurocomputing.

[4]  Mohamed Elhoseny,et al.  Deep neural network-based application partitioning and scheduling for hospitals and medical enterprises using IoT assisted mobile fog cloud , 2021, Enterp. Inf. Syst..

[5]  Ning Ge,et al.  Failure prediction in production line based on federated learning: an empirical study , 2021, Journal of Intelligent Manufacturing.

[6]  Sukhpal Singh Gill,et al.  IoT and Fog-Computing-Based Predictive Maintenance Model for Effective Asset Management in Industry 4.0 Using Machine Learning , 2021, IEEE Internet of Things Journal.

[7]  Raed Abdel Sater,et al.  A Federated Learning Approach to Anomaly Detection in Smart Buildings , 2020, ACM Trans. Internet Things.

[8]  Taha Landolsi,et al.  Scheduling Internet of Things requests to minimize latency in hybrid Fog-Cloud​ computing , 2020, Future Gener. Comput. Syst..

[9]  Reza M. Parizi,et al.  Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications , 2020, IEEE Access.

[10]  M. Shamim Hossain,et al.  Deep Anomaly Detection for Time-Series Data in Industrial IoT: A Communication-Efficient On-Device Federated Learning Approach , 2020, IEEE Internet of Things Journal.

[11]  Adam A. Alli,et al.  The fog cloud of things: A survey on concepts, architecture, standards, tools, and applications , 2020, Internet Things.

[12]  Jun Zhang,et al.  Communication-Efficient Edge AI: Algorithms and Systems , 2020, IEEE Communications Surveys & Tutorials.

[13]  Tim Verbelen,et al.  A Survey on Distributed Machine Learning , 2019, ACM Comput. Surv..

[14]  Nicola Dragoni,et al.  Foundations and Evolution of Modern Computing Paradigms: Cloud, IoT, Edge, and Fog , 2019, IEEE Access.

[15]  Wei Yang Bryan Lim,et al.  Federated Learning in Mobile Edge Networks: A Comprehensive Survey , 2019, IEEE Communications Surveys & Tutorials.

[16]  Zhiwu Huang,et al.  Remaining Useful Life Estimation Using CNN-XGB With Extended Time Window , 2019, IEEE Access.

[17]  Giuseppe Aceto,et al.  A Survey on Information and Communication Technologies for Industry 4.0: State-of-the-Art, Taxonomies, Perspectives, and Challenges , 2019, IEEE Communications Surveys & Tutorials.

[18]  Kin K. Leung,et al.  Energy-Efficient Radio Resource Allocation for Federated Edge Learning , 2019, 2020 IEEE International Conference on Communications Workshops (ICC Workshops).

[19]  S. H. Song,et al.  Client-Edge-Cloud Hierarchical Federated Learning , 2019, ICC 2020 - 2020 IEEE International Conference on Communications (ICC).

[20]  Qiang Yang,et al.  Federated Machine Learning , 2019, ACM Trans. Intell. Syst. Technol..

[21]  Anit Kumar Sahu,et al.  Federated Optimization in Heterogeneous Networks , 2018, MLSys.

[22]  Mats Jirstrand,et al.  A Performance Evaluation of Federated Learning Algorithms , 2018, DIDL@Middleware.

[23]  Kaibin Huang,et al.  Towards an Intelligent Edge: Wireless Communication Meets Machine Learning , 2018, ArXiv.

[24]  Qin Zhang,et al.  Edge Computing in IoT-Based Manufacturing , 2018, IEEE Communications Magazine.

[25]  Jae-Yoon Jung,et al.  LiReD: A Light-Weight Real-Time Fault Detection System for Edge Computing Using LSTM Recurrent Neural Networks , 2018, Sensors.

[26]  Kin K. Leung,et al.  When Edge Meets Learning: Adaptive Control for Resource-Constrained Distributed Machine Learning , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[27]  Xiang Li,et al.  Remaining useful life estimation in prognostics using deep convolution neural networks , 2018, Reliab. Eng. Syst. Saf..

[28]  Amit P. Sheth,et al.  Machine learning for Internet of Things data analysis: A survey , 2017, Digit. Commun. Networks.

[29]  Kay Chen Tan,et al.  Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Matthias Eberl,et al.  Cloud, fog and edge: Cooperation for the future? , 2017, 2017 Second International Conference on Fog and Mobile Edge Computing (FMEC).

[31]  Peter Richtárik,et al.  Federated Optimization: Distributed Machine Learning for On-Device Intelligence , 2016, ArXiv.

[32]  Xiaoli Li,et al.  Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life , 2016, DASFAA.

[33]  Jakub Konecný,et al.  Federated Optimization: Distributed Optimization Beyond the Datacenter , 2015, ArXiv.

[34]  Jacques Bughin,et al.  The internet of things: mapping the value beyond the hype , 2015 .

[35]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[36]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[37]  Yuan Gao,et al.  A survey on federated learning , 2021, Knowl. Based Syst..

[38]  Sourabh Bharti,et al.  Privacy-Aware Resource Sharing in Cross-Device Federated Model Training for Collaborative Predictive Maintenance , 2021, IEEE Access.

[39]  Marisa Mohr,et al.  Towards Collaborative Predictive Maintenance Leveraging Private Cross-Company Data , 2020, GI-Jahrestagung.